When you read the drupal_get_filename() function, you can see it was design to always have a fallback when the caller asks for an unknown file, which does a full FS scan for finding potential matching .info files.
This is OK for cases where the database is not there or when debugging, but on a normal use case (I.E. production site with normal users hits) this is resource intensive, it can happen at normal runtime.
I/O are one the biggest bottleneck in web development, and should be avoided. If a file can't be find in static cache while running normally, no such intensibe I/O should be made.
Typically, on my environment, using drupal 7 beta 3, right after upgrading a simple D6 site to D7, each hit on the admin path triggered 84 calls to this function (attempting to load modules I manually removed because not ported to D7), almost 80% of these calls triggered the FS scan, and each page in the admin section took something like 5 seconds to build (by disabling the full FS scan from the function, in the same environment, the execution takes between 500 and 600 ms).
Another pitfall is that the method actually does one SQL request each time its being called (and file has not already been requested btw), and it can be called really a lot in the same page hit. This is not acceptable to have one hundred SQL calls on each page when it could be reduced to only one quite easily. D7 is intensive for database, and PDO is a lot slower than mysql_* and pg_* PHP extensions, so we/you should reduce those queries, it would be a great benefit for overall performances.
I think that in case the DB layer is active, the drupal_get_filename() function should never attempt to do a FS scan, even in case a file is not being found in static/database case, and leave that only at cache clear time or when the modules administration page is hit by a site administror.
What do you think of this ? I really think it would be a great optimization. This is not much on the overall Drupal runtime, but reducing in some case 100 queries to only 1, and avoid I/O would greatly boost a lot of things.
Remember that Drupal can be installed on many environments, and some could be on distant filesystem, mounted using NFS or used over a distant NAS storage, and an FS scan in those conditions is really not doable.
Comments
Comment #1
pounardHere is a sample implementation (that needs work) which would fit to avoid some I/O:
Comment #2
XiaN Vizjereij commentedSubscribing
Comment #3
chx commentedI am not sure there is a bug here. system_list primes drupal_get_filename, on admin/ the install files will indeed be scanned for yes but that's not a page where we care about performance.
Comment #4
pounardWe should care about all pages performances, everyday admin tasks might be done on a production site.
I'm going to do some more debuging, to ensure there is no other way to trigger this file scan.
The system_list() prime might be documented in the drupal_get_filename() function I think, it would be great for people that read this code.
Comment #5
pounardOk, so this is a false alert, the behavior I encountered on beta3 seems to have been resolved using RC1. I did some debugging, drupal_get_filename() is being called 84 times on my custom site, most of the calls will set a new file in cache thanks to system_list() function as chx said. There is still 42 calls to file_exists() which is not that bad (but may still be avoided).
Comment #6
bojanz commentedIf I am reading #3 and #5 correctly, looks like we're good.
Comment #7
pounardWe're good.
Comment #8
pounardWhy did the bugtracker reopened the bug?.. Reclosing it.