Problem/Motivation

Follow-up to #2291449: Add Twig template inheritance based on the theme registry, enable adding Twig loaders.

@Fabianx spotted the file_exists() when we are registering namespaces in \Drupal\Core\Template\Loader\FilesystemLoader.

Proposed resolution

The file_exists() is not necessary here, just remove it. This would make all modules and themes available as Twig namespaces regardless of whether a 'templates' directory exists in the module/theme.

Remaining tasks

Patch

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Task because it's a performance enhancement.
Issue priority Major because we should not have any file_exists() calls during runtime.
Unfrozen changes Unfrozen because it is a performance improvement in the theme system.
Disruption No disruption.

User interface changes

n/a

API changes

n/a

Files: 
CommentFileSizeAuthor
#28 interdiff.txt612 bytesCottser
#28 2383413-28.patch1.04 KBCottser
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 87,007 pass(es). View

Comments

Cottser’s picture

Issue summary: View changes
Status: Active » Postponed
Cottser’s picture

Although this might cause some issues (in \Twig_Loader_Filesystem) :/

if (!is_dir($path)) {
  throw new Twig_Error_Loader(sprintf('The "%s" directory does not exist.', $path));
}
Cottser’s picture

I suppose we could add our own version of addPath() that removes that :)

Cottser’s picture

Status: Postponed » Active
alvar0hurtad0’s picture

Status: Active » Needs review
FileSize
662 bytes
FAILED: [[SimpleTest]]: [PHP 5.4 MySQL] 47,489 pass(es), 26,289 fail(s), and 7,264 exception(s). View

I suppose it is.

Status: Needs review » Needs work

The last submitted patch, 5: remove_file_exists-2383413-5.patch, failed testing.

joelpittet’s picture

#2 seems to have been spot on;) Thanks for testing that theory out @alvar0hurtad0

alvar0hurtad0’s picture

Ummmmm

what should I do?

submit the patch again or let it be?

joelpittet’s picture

Assigned: Unassigned » Cottser

@Cottser I'm assigning to you, but maybe this is something for @Fabianx to have a look at and provide suggestions.

It looks like either an upstream change or maybe a try/catch maybe? All guesses.

@alvar0hurtad0 we'll see what they have to say before going unless you have some ideas you want to try, be my guest.

Cottser’s picture

Assigned: Cottser » Unassigned

I suggest #3, we are already subclassing the Twig Filesystem class, we can override addPath to remove the is_dir check. As long as that has no nasty side effects it seems like the best way forward to get the performance we want.

Make sense?

joelpittet’s picture

Would there be any side effects to loading paths into the loader that don't exist?

larsmw’s picture

FileSize
1.15 KB
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 86,704 pass(es). View

I have made a patch for overriding addPath in FilesystemLoader.

tadityar’s picture

Status: Needs work » Needs review

Setting to needs review for patch to be tested.

joelpittet’s picture

Status: Needs review » Needs work

This seems like a good solution I think... just a bit of cleanup needed here.

  1. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,26 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +      /**
    +     * Adds a path where templates are stored.
    +     *
    +     * @param string $path      A path where to look for templates
    +     * @param string $namespace A path name
    +     *
    +     * @throws Twig_Error_Loader
    +     */
    +    public function addPath($path, $namespace = self::MAIN_NAMESPACE)
    +    {
    +        if (!is_dir($path)) {
    +            return FALSE;
    +        }
    

    This should use drupal coding standards for indents. IE 2 spaces.

  2. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,26 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +        // invalidate the cache
    +        $this->cache = array();
    

    I'd Leave this above to still invalidate the cache even if it's not a dir, just like upstream does.

larsmw’s picture

Isn't this supposed to throw an exception instead of return FALSE? I just dont know which exception!?

joelpittet’s picture

Not from the issue summary, but maybe it should? @Cottser thoughts?

Cottser’s picture

I think the idea would be to just add all directories regardless of whether they exist or not, we can double check to see how Twig handles it when trying to load namespaced templates to see if it causes any issues. But right now I don't see any other way forward if we want to remove the file_exists().

Hopefully that makes sense, it's early :)

larsmw’s picture

FileSize
1 KB
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 86,755 pass(es). View

Cottser, you mean like this?

joelpittet’s picture

Status: Needs work » Needs review

Needs review status will trigger testbot to check the patch.

  1. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,21 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +  /**
    +  * Adds a path where templates are stored.
    

    The commenting of the * lines need to be indented one space.

  2. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,21 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +  * @param string $path A path where to look for templates
    ...
    +    // invalidate the cache
    

    The end of the lines should end in a Period and Start with a capitol.

  3. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,21 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +  * @param string $namespace A path name
    +  *
    +  */
    

    This line break is not need.

larsmw’s picture

FileSize
1 KB
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 86,758 pass(es). View

I have cleaned it up now.

Cottser’s picture

Status: Needs review » Needs work

@larsmw, yes to #18 - code looks good now, thanks! Please post interdiffs when you update the patch, great habit to get into early and makes it easier to review changes. Getting patches reviewed is most often the biggest bottleneck in the issue queues so anything you can to do help make that easier is a huge win in my opinion :)

  1. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,19 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +   * @param string $path A path where to look for templates
    +   * @param string $namespace A path name
    

    These should be updated to match our documentation standards on params, whitespace and punctuation changes would be needed: https://www.drupal.org/node/1354#param

  2. +++ b/core/lib/Drupal/Core/Template/Loader/FilesystemLoader.php
    @@ -43,10 +43,19 @@ public function __construct($paths = array(), ModuleHandlerInterface $module_han
    +  }
     }
    

    Minor: Missing blank line between the last method and the end of the class per https://www.drupal.org/node/608152#indenting.

joelpittet’s picture

What is the reason behind adding directories to the registry that may not exist? AKA can we leave the is_dir check in for parity upstream?

Cottser’s picture

It's kind of a cascade, the is_dir() will fail without the file_exists() filtering out nonexistent directories. The nonexistent directories are just being added as possible namespaces, Twig should still complain on its own when you try to extend/include/whatever a template that doesn't exist, that could use a bit of manual (or automated?) testing.

joelpittet’s picture

Ah, I'm thinking from Twig's perceptive having to look in folders that don't exist when searching for templates, though I guess once it's found the template it doesn't have to search through these folders again. As long as storing these potentially empty templates doesn't have an negative effect on performance as the IS is intending to solve, it's cool with me;)

Sensitive to file searching due to module_load_include issues last week in D7. #1443308: Add static cache to module_load_include()

Cottser’s picture

Because we have the string loader, calling a nonexistent template just prints the string, with or without the patch. So no concerns there. This also means that I'm pretty sure we don't have a good way of testing this currently (like by catching an exception) until #2369981: Not found templates are displayed literally instead of throwing an Exception due to string loader is resolved, which I just bumped to major.

larsmw’s picture

FileSize
1.03 KB
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 86,892 pass(es). View
721 bytes

Implemented comments from #21

larsmw’s picture

Status: Needs work » Needs review
Cottser’s picture

Status: Needs review » Reviewed & tested by the community
FileSize
1.04 KB
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 87,007 pass(es). View
612 bytes

Attaching one minor change to remove a now unnecessary variable. This looks good to me now as far as documentation and coding style, and solves the stated task, RTBC. I don't think this needs additional test coverage.

--

For this one profiling on an SSD is probably not the best way to test, but I was able to confirm that we are cutting down on calls to file_exists() (and is_dir()).

Scenario: Standard profile, Bartik homepage, added 17 search blocks to bring the total of search blocks up to 18. Gave anonymous users the 'search content' permission.

Reduced function calls:

file_exists(): -45
is_dir(): -25

There are 21 more calls to rtrim() with this approach, but those are extremely cheap.

=== 8.0.x..8.0.x compared (54e0a6d45fe2a..54e0a77153dbf):

ct  : 166,652|166,652|0|0.0%
wt  : 698,622|696,872|-1,750|-0.3%
cpu : 682,564|681,253|-1,311|-0.2%
mu  : 49,588,128|49,588,128|0|0.0%
pmu : 50,180,616|50,180,616|0|0.0%

http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=54e0a6d45fe2a&...

=== 8.0.x..twig-ns-file-exists-2383413-26 compared (54e0a6d45fe2a..54e0a7d2151b1):

ct  : 166,652|166,624|-28|-0.0%
wt  : 698,622|697,883|-739|-0.1%
cpu : 682,564|682,200|-364|-0.1%
mu  : 49,588,128|49,597,752|9,624|0.0%
pmu : 50,180,616|50,191,232|10,616|0.0%

http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=54e0a6d45fe2a&...

webchick’s picture

Assigned: Unassigned » catch

Can't think of a reason we would not want to do this, but booting over to catch since it touches performance stuff.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Looks great. Committed/pushed to 8.0.x, thanks!

  • catch committed 70e4656 on 8.0.x
    Issue #2383413 by larsmw, Cottser, alvar0hurtad0: Remove file_exists()...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.