Allow file_scan_directory() to ignore folder

Interested in providing a patch for this?

Thanks @kolier, this seems like a good idea for folders you mentioned in the issue summary:) I currently don't have the use-case but I'll give it a try when I do. Heads up, it may get bumped to D8 before it gets in D7, though I'm going to keep the version here and let someone else bump it if they see it fitting.

Another option is to change the default "nomask" option to include node_modules and other folders that do not need to be scanned by default.

Nothing goes into D7 without going into D8 first.

Looks like my fix won't work for D8 as the nomask option/default mask does not exist. I haven't had time to test if file_scan_directory descends into node_modules (and others) by default, but I'd guess they would.

(Here is the D7 version)

The nomask option still exists, there is just no default for it as of #1165694: Remove default nomask from file_scan_directory()

https://github.com/drupal/drupal/blob/8.0.x/core/includes/file.inc#L1052 now just checks first.

Yeah, adding "node_modules" and "bower_components" (others?) as a default would need to happen where Drupal does it default file scanning (which I haven't tracked down for D8). In my opinion, Drupal does not need to search those directories by default if they are present and in D7 it can significantly affect performance when they are present in the Drupal root. I think this will become more common as Node, Bower, Grunt, Gulp, etc. become more common tools for Drupal developers. In the case on a recent project, this affects our dev environments but not production because we're not installing these things there. @kolier patch would also work for ignoring these directories but requires adding the .scanignore file in all these locations.

I'll try to test this with D8 soon. For [my] reference, here's things calling file_scan_directory in D8.

From my initial testing I strongly feel this is only a D7 thing and may only affect D8 if these directories are in a theme. Out of the box, D7 is much more aggressive with scanning the filesystem than D8 (at least with file_scan_directory, not sure about other things). One thing to double check in D7: I think node_modules (and others) may be ignored if they are at the Drupal root, but are not when they are in the theme or in a module.

FYI I put this in my node_modules and bower_components with

touch .scanignore

On cache flush:

Before:

file_scan_directory
Calls: 170 @ 5.5%
Wall Time: 1,430,240 @ 6.6%
CPU Time: 1,401,182 @ 8.0%
Mem: 672,672 @ 0.2%
Peak Mem: 23,904 @ 0.0%

After:

file_scan_directory
Calls: 170 @ 5.2%
Wall Time: 193,286 @ 1.0%
CPU Time: 120,588 @ 0.8%
Mem: 514,072 @ 0.4%
Peak Mem:36,128 @ 0.0%

This bit me pretty hard. I've added the following to my package.json to generate the .scanignore file when 'npm install' is run.

"scripts": {
  "install": "echo 'https://www.drupal.org/node/2393605' > node_modules/.scanignore"
}

I've added the link inside the file so others have some way to figure out why it's there.

@ZHawkins awesome idea! Especially because node_modules and bower packages are not usually committed to the repo! Brilliant!

Adding related issue #2329453: Ignore front end vendor folders to improve directory search performance

I like this idea as a solution to #2329453: Ignore front end vendor folders to improve directory search performance, but I'm not sure about the filename .scanignore. Even though Google doesn't return any results for that filename at the moment, I think we should choose a name that is unlikely to cause collisions with other software that happens to use the same name. I would prefer something namespaced like .drupalignore.

.drupal_scan_ignore? Theoretically, other scenarios may arise where Drupal is required to ignore other things but for a different reason. This would namepsace both the software and the purpose/context.

This hit me hard.. https://twitter.com/dcorbacho/status/632132568587051008
Not only me https://twitter.com/crizzirc/status/632138118695682049
This solution proposed sounds very good idea and clean. ONCE that you know this problem.
+1 to marcvangend idea to use ".drupalignore"

But somehow, I must agree with mark carver on #2329453: Ignore front end vendor folders to improve directory search performance where he thinks this would be "another drupalism".
If someone new to Drupal runs npm install on a Drupal folder, and starts to slow down the site.. and he's not aware of this issue, he will just blame Drupal for being slow. Front end developers shouldn't need to be aware of how Drupal core works internally. (by default)

This is a duplicate of #2329453: Ignore front end vendor folders to improve directory search performance which is committed to 8.x. Closing so people aren't confused.

For anyone looking for the D7 backport of the other issue, it was moved to #2482549: Ignore node_module folder in core to use Drupal with npm/grunt/nodejs.