• Replaces require_once with include_once making it more robust to module erroneous calls.
  • Errors due to missing files are shown as warnings: better warn the module developers that they did an error than letting them silent.
  • Removed the surrounding if(is_file()) making this code 100% I/O free when used with an opcode cache. Both PHP vanilla and opcode caches already optimize include statements for us, checking for the file existence is bypassing this optimization.

Comments

pounard’s picture

Status: Active » Needs review
Issue tags: +Performance
StatusFileSize
new469 bytes

Status: Needs review » Needs work
Issue tags: -Performance

The last submitted patch, 1978592-1-module_load_include-is_file.patch, failed testing.

pounard’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work
Issue tags: +Performance

The last submitted patch, 1978592-1-module_load_include-is_file.patch, failed testing.

soul88’s picture

Priority: Normal » Major

Got the same problem with Commons distribution. waiting for is_file to complete takes 15-20% of page load time on the most heavy pages (ex.: groups)

soul88’s picture

Priority: Major » Normal

seems that it's strongly related to this: https://drupal.org/node/752730

soul88’s picture

Status: Needs work » Closed (duplicate)
pounard’s picture

Status: Closed (duplicate) » Needs work

There is multiple reasons why this is not a duplicate:

  • This patch focuses on module_load_include() only, because it's a very small change and can be safely done without being extensive over the system
  • Drupal 7 and Drupal 8 are very different, the amalgam cannot be made: other issues have been switched to Drupal 8
pounard’s picture

Ok spoke too quickly, closing in flavor of #1524630: replace is_file() + require_once() with include_once() for apc performance.

I remebered now why I opened this bug, I was already following the others, and most of them have either been switched to D8, either being closed under other various assumptions. I read them all, I participated in most of them, and I opened this one in order to restrict the field of application of the global idea and make a less extensive, less intrusive although still efficient patch.

pounard’s picture

Status: Needs work » Needs review
Issue tags: -Performance

Status: Needs review » Needs work
Issue tags: +Performance

The last submitted patch, 1978592-1-module_load_include-is_file.patch, failed testing.

pounard’s picture

StatusFileSize
new470 bytes

I'm not fond of the @ operator but Drupal always attempt to load non existing files, which is a deeper problem. Thus the operator is mandatory for this very simple patch to work.

pounard’s picture

Status: Needs work » Needs review
pounard’s picture

Issue tags: +Drupal 7 is not dead yet

Yay green! Please anyone reviews :)

soul88’s picture

Issue tags: +commonslove
StatusFileSize
new945 bytes

I'd like to propose a bit more extended approach. I'm now working on Drupal Commons site and on some pages I have 1700+ calls of this function on the page. And only 34 of them have unique parameters. So I'd like to propose to add static cache additionally to the patch above. It saves me 0,3-1 sec per page load.

soul88’s picture

Some details can be found here: https://drupal.org/node/2077375 (XHProf listings mostly)

Status: Needs review » Needs work

The last submitted patch, 1978592-15-module_load_include-performance.patch, failed testing.

soul88’s picture

reroll

soul88’s picture

Status: Needs work » Needs review

changed status

Status: Needs review » Needs work

The last submitted patch, 1978592-17-module_load_include-performance.patch, failed testing.

soul88’s picture

soul88’s picture

Status: Needs work » Needs review
pounard’s picture

I actually did test the exact opposite, removing this kind of static cache from a few functions in core, and with APC it was quicker without, plus PHP consumed less memory. A decent OPcode cache will already do this static cache for you but at a lower level, and will be faster than you. I'd be curious to know if any is an OPcode cache enabled on your testing box?

pounard’s picture

1700+ calls

This sound huge! I'd say you have a problem on your site. If you are using a basic commons distribution without modifications, then I'd say Commons has never been designed to perform great.

soul88’s picture

replied in the post about Commons. APC is installed.

https://drupal.org/node/2077375

pounard’s picture

I will do some benchmarks on my side installing a Drupal commons with Drupal vanilla, my patch and your patch (if I have time on multiple different boxes). Nevertheless I also will try to see why this function is being called so many time, it sounds like a bug to me.

Sylvain_G’s picture

The patch on #21 causes Fatals error with i18n

https://drupal.org/node/2082573#comment-7832895

pounard’s picture

pounard’s picture

Issue summary: View changes
StatusFileSize
new470 bytes
pounard’s picture

This actually works, please I need some reviews here.

soul88’s picture

Status: Needs review » Reviewed & tested by the community

The last patch works great for me.

pounard’s picture

Status: Reviewed & tested by the community » Needs review

I'm not confortable with proceeding with this without profiing a variety of different sites, some more reviews and some benchmarks/profiling traces would be helpful too! Thanks for testing.

soul88’s picture

On my site I've got 70-110 "is_file" calls on the page that take 30-50ms. It's about 2% of page load time.
//PHP5.5 + cache opcode enabled

joelpittet’s picture

This seems like a better and maybe more comfortable solution for D7. #1443308-9: Add static cache to module_load_include()

@pounard maybe close as duplicate if you agree?

pounard’s picture

I'm definitely not really happy with polluting Drupal with "Yet another static cache" whereas the optimization on that specific include call should be done by the VM or the OPCode, not us. In the other issue, did you do the same benchmarks as those exposed with a single include_once without static cache and without file_exists() ?

joelpittet’s picture

In the other issue I just tested before and after the patch.

I researched a bit alternatives but they all seem to have drawbacks with OPcode caching so I really didn't pay too much attention to them.

pounard’s picture

"Seem" does not seem legit to me :) I'd be curious to see how it behave both of:
https://www.drupal.org/files/issues/1978592-29-module_load_include-is_fi... (#29)
and https://www.drupal.org/files/1978592-1-module_load_include-is_file.patch (#1)
compared to doing a static cache.

joelpittet’s picture

Issue tags: +needs profiling

Then you know what to do;) I don't see that being a big improvement so not interested in profiling this ATM. Feel free to see what gains you get.

pounard’s picture

Yes I agree I should probably do some benchmarks myself, will do if I find some time to do it.

pounard’s picture

Status: Needs review » Closed (duplicate)

@Joelpittet did it, and https://www.drupal.org/node/1443308#comment-10688528 so I close this issue.