Just by turning on twig debug and looking at theme suggestions, you can see for the main menu (and any corresponding menu like footer) that there is a duplicate theme suggestion and at the most specific level which overrides any and all customizations in hook_theme_suggestions_alter().

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'menu__main' -->
<!-- FILE NAME SUGGESTIONS:
   x menu--main.html.twig
   * menu--main--customization.html.twig
   x menu--main.html.twig
   * menu.html.twig
-->

Should be:

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'menu__main' -->
<!-- FILE NAME SUGGESTIONS:
   x menu--main--customization.html.twig
   * menu--main.html.twig
   * menu.html.twig
-->

Issue fork drupal-2752443

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

sylus created an issue. See original summary.

sylus’s picture

Issue summary: View changes
sylus’s picture

Title: Filename Suggestions aren't correct for menu's » Filename suggestions aren't correct for menu theme hook
Issue summary: View changes
sylus’s picture

Title: Filename suggestions aren't correct for menu theme hook » Filename suggestions duplicated for menu theme hook in Bootstrap Theme
Project: Drupal core » Bootstrap
Version: 8.1.x-dev » 8.x-3.x-dev
Component: menu system » Code

Moving this to bootstrap as couldn't replicate on Drupal Core but as soon as switched to bootstrap theme, then this issue popped up.

markhalliwell’s picture

Priority: Normal » Minor

Feel free to try and figure out what's going on there. IMO this really isn't that big of a deal.

sylus’s picture

The `menu--main.html.twig` is always the most specific suggestion, so I can't leverage the `menu--main--customization.html.twig` theme suggestion ever, as the regular one is always superceding. This breaks the proper theming workflow and makes it impossible to do custom menu's.

Strangely this only occurs for menu suggestions and doesn't affect any others like blocks.

Will try to figure out what is causing this but thanks for taking the time to comment :)

markhalliwell’s picture

The `menu--main.html.twig` is always the most specific suggestion, so I can't leverage the `menu--main--customization.html.twig` theme suggestion ever, as the regular one is always superceding. So this breaks the proper theming workflow.

No, that's not how theme suggestions work. They work from the bottom up and stop at the first matching one. Your own code proves this (see where the X is):

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'menu__main' -->
<!-- FILE NAME SUGGESTIONS:
   * menu--main.html.twig
   x menu--main--customization.html.twig <--- this is what is used
   * menu--main.html.twig
   * menu.html.twig
-->
sylus’s picture

Apologies I actually wrote those theme suggestions based on memory so the 'X' was wrong here is a recent cut/paste. You can see that both are active and never takes my most specific menu template.

<!-- FILE NAME SUGGESTIONS:
   x menu--main.html.twig
   * menu--main--wxt-bootstrap-main-menu--wet-boew.html.twig
   * menu--main--wxt-bootstrap-main-menu.html.twig
   x menu--main.html.twig
   * menu.html.twig
-->
<!-- THEME DEBUG --><!-- THEME HOOK: 'menu__toolbar__admin' --><!-- FILE NAME SUGGESTIONS:
   * menu--toolbar--admin.html.twig
   x menu--toolbar.html.twig
   x menu--toolbar.html.twig
   * menu.html.twig
-->

I've updated the OP with the actual code copied.

sylus’s picture

Issue summary: View changes
sylus’s picture

Issue summary: View changes
StatusFileSize
new71.49 KB

Attaching an image showing the templates I have including `menu--main--wxt-bootstrap-main-menu--wet-boew.html.twig` which is the one I want to take hold.

Menu

sylus’s picture

sylus’s picture

Issue summary: View changes
sylus’s picture

I read your most recent comment stating it goes from the bottom up and stops on the first matching pattern. If that were the case wouldn't it only ever match menu.html.twig?

markhalliwell’s picture

Title: Filename suggestions duplicated for menu theme hook in Bootstrap Theme » Duplicate theme hook suggestions in Twig debug output
Project: Bootstrap » Drupal core
Version: 8.x-3.x-dev » 8.2.x-dev
Component: Code » theme system
Issue tags: +Needs backport to D7

Sorry, I meant top down.

So, I've actually been stepping through this. From what I can tell this is, in fact, a core "bug".

Looking through \Drupal\Core\Theme\ThemeManager::render, I can see that the suggestions provided are actually correct. Where the "break down" actually occurs is in twig_render_template, specifically with the twig_debug suggestion output. Basically, it just merges suggestions without first seeing if they are already in the array. This can cause duplicates to appear if the suggestions were derived from splitting whatever was passed to #theme using the __ notation.

So this is really just a display issue, it doesn't actually affect any of the logic in which theme hook suggestion is chosen.

I'll post a patch here shortly.

markhalliwell’s picture

Title: Duplicate theme hook suggestions in Twig debug output » In correct order and duplicate theme hook suggestions
Priority: Minor » Major
Issue tags: +Needs tests

Actually, this is a pretty major bug after all.

It turns out that \Drupal\Core\Theme\ThemeManager::render isn't doing what it's supposed to be doing after all. Even with the duplicates removed, the menu__toolbar__admin suggestion should be appearing at the top. It is not.

This is because the original theme hook that was passed to #theme is never actually added to the $suggestions array. This means that if someone actually implemented menu--toolbar--admin.html.twig, it's theme registry info wouldn't be used, only the menu__toolbar would be.

<!-- FILE NAME SUGGESTIONS:
   x menu--toolbar.html.twig
   * menu--toolbar--admin.html.twig
   * menu.html.twig
-->
markhalliwell’s picture

Status: Active » Needs review
StatusFileSize
new2.65 KB

Status: Needs review » Needs work

The last submitted patch, 16: in_correct_order_and-2752443-1.patch, failed testing.

sylus’s picture

Just chiming in here that the latest patch does indeed solve the problem and the appropriate template is now being called.

Thank you so much to @markcarver for the detailed inspection and analysis of the problem. ^_^

Here is the test currently failing though might be a false positive as the order being checked seems wrong:

// Create another node and make sure the template suggestions shown in the
    // debug markup are correct.
    $node3 = $this->drupalCreateNode();
    $build = array('#theme' => 'node__foo__bar');
    $build += node_view($node3);
    $output = $renderer->renderRoot($build);
    $this->assertTrue(strpos($output, "THEME HOOK: 'node__foo__bar'") !== FALSE, 'Theme call information found.');
    $this->assertTrue(strpos($output, '* node--foo--bar' . $extension . PHP_EOL . '   * node--foo' . $extension . PHP_EOL . '   * node--&lt;script type=&quot;text/javascript&quot;&gt;alert(&#039;yo&#039;);&lt;/script&gt;' . $extension . PHP_EOL . '   * node--3--full' . $extension . PHP_EOL . '   * node--3' . $extension . PHP_EOL . '   * node--page--full' . $extension . PHP_EOL . '   * node--page' . $extension . PHP_EOL . '   * node--full' . $extension . PHP_EOL . '   x node' . $extension) !== FALSE, 'Suggested template files found in order and base template shown as current template.');

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

davy-r’s picture

The patch doesn't completely fix the issue.

Test 1

Result: FAIL, wrong order of displayed suggestions

Available templates: "input.html.twig"
Rendered template: "input.html.twig"
Debug output:

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'input__search' -->
<!-- FILE NAME SUGGESTIONS:
   * input--search.html.twig
   * input--search--search-block-form.html.twig
   x input.html.twig
-->
<!-- BEGIN OUTPUT from 'core/themes/stable/templates/form/input.html.twig' -->

Test 2

Result: OK

Available templates: "input.html.twig", "input--search.html.twig"
Rendered template: "input--search.html.twig"
Debug output:

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'input__search' -->
<!-- FILE NAME SUGGESTIONS:
   * input--search--search-block-form.html.twig
   x input--search.html.twig
   * input.html.twig
-->
<!-- BEGIN OUTPUT from 'themes/custom/themexyz/templates/input/input--search.html.twig' -->

Test 3

Result: OK

Available templates: "input.html.twig", "input--search.html.twig", "input--search--search-block-form.html.twig"
Rendered template: "input--search--search-block-form.html.twig"
Debug output:

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'input__search' -->
<!-- FILE NAME SUGGESTIONS:
   x input--search--search-block-form.html.twig
   * input--search.html.twig
   * input.html.twig
-->
<!-- BEGIN OUTPUT from 'themes/custom/themexyz/templates/input/input--search--search-block-form.html.twig' -->
davy-r’s picture

Status: Needs work » Needs review
StatusFileSize
new1.04 KB

Tried a different approach in a attempted to resolve the issue. I don't have in-depth knowledge about the theme_hook_suggestions, but this seems to be resolving the issue in my case..

Status: Needs review » Needs work
davy-r’s picture

Status: Needs work » Needs review
StatusFileSize
new4.83 KB

This morning I've been working though the process of determining the file name suggestions.

ThemeManager
#1: hook_theme_suggestions_HOOK is adding the "base" suggestions to $suggestions.
#2: When the theme implementation was invoked with a direct theme suggestion (like '#theme' => 'node__foo__bar') AND there is a template for that suggestion, then that suggestion is added to $suggestions.
#3: hook_theme_suggestions_HOOK_alter and hook_theme_suggestions_alter may modify the $suggestions, and add any suggestions before and/or after the current $suggestions.
#4: The $suggestions array is reversed and the first template suggestion that exists as a template, is being set as the template_file that is used for rendering.
Twig engine
- Attempts to redetermine the suggestions and print these as the file name suggestions.

In the attached patch I build a complete array of $suggestions, which always includes the suggestions from direct theme suggestion (like '#theme' => 'node__foo__bar'). Just before rendering I add the $suggestions to the $variables, to make sure we pass the file name suggestions that were actually used. In the Twig engine we can now just print the file name suggestions as they where actually used (instead of trying to redetermine them).

edit: ThemeSuggestionsAlterTest failing, will have a look on that later this week..

Status: Needs review » Needs work
davy-r’s picture

Status: Needs work » Needs review
StatusFileSize
new5.32 KB
davy-r’s picture

It turned out that the base suggestion should always have the lowest priority.

markhalliwell’s picture

Title: In correct order and duplicate theme hook suggestions » Incorrect order and duplicate theme hook suggestions
Status: Needs review » Needs work

Patch from #26 seems to be working just fine. Setting back to CNW for tests of the OP.

acbramley’s picture

I'm not sure if this is related but I've found another issue to do with the base themes used in ThemeManager::alterForTheme where if you have the following theme structure:

- foo
-- bar
--- baz

And foo and bar both implement hook_theme_suggestions_alter, the suggestions from foo will appear after the suggestions from bar.

We have this issue with a subtheme of bootstrap that has some templates for form inputs, then site specific subthemes of that are adding their own templates but they are not being used. Other places in the registry reverse that base themes array before iterating over it so maybe that's also causing issues here?

lauriii’s picture

Issue tags: +Triaged core major

@Cottser, @alexpott, @xjm, @joelpittet, and I discussed this and agreed that this is a major bug because this bug causes theme suggestions to work in a very unpredictable way. There is also no known workaround for this bug.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

sylus’s picture

Just adding a related issue that is also addressing same area of code.

mian3010’s picture

Attached new patch that applies to 8.3.x

davy-r’s picture

sudhanshug’s picture

Status: Needs work » Needs review

Changing status to queue the last submitted(from #33) patch for tests

sylus’s picture

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

gambry’s picture

Status: Needs review » Needs work

Thanks everyone for the work done so far!

  1. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -155,14 +155,19 @@ public function render($hook, array $variables) {
    +      foreach (array_slice($derived_suggestions, 1) as $hook) {
    

    A comment explaining why we slice with offset 1 can help the reader. I'm assuming that's because element in position 0 is always the original hook ($hook = $original_hook = $derived_suggestion), and so already checked by previous condition if (!$theme_registry->has($hook)){}? (and if so, do we need array_slice() at all? Suggestion: why don't we removed the first if (!$theme_registry->has($hook)){} and the array_slice() at all?)

  2. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -228,12 +233,14 @@ public function render($hook, array $variables) {
    +    if (isset($info['base hook']) && !in_array($hook, array_slice($derived_suggestions, 0, -1))) {
           $suggestions[] = $hook;
    ...
    +    $suggestions = array_merge($suggestions, array_reverse(array_slice($derived_suggestions, 0, -1)));
    

    We now slice omitting last element of the array. Again, maybe a comment describing why we do it would benefit the reader. Also splitting the 2 condition may help readability, as my understanding is they don't relate each other?

    Also, why do we array_reverse() the slice of derived_suggestions? aren't suggestions already in the right specificity order (more -> less)?

  3. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -244,6 +251,11 @@ public function render($hook, array $variables) {
    +    // Add the base suggestion, should always have the lowest priority
    

    Missing period.

  4. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -244,6 +251,11 @@ public function render($hook, array $variables) {
    +    if (!in_array(array_slice($derived_suggestions, -1), $suggestions)) {
    +      $suggestions = array_merge(array_slice($derived_suggestions, -1), $suggestions);
    +    }
    

    This logic is probably wrong. It doesn't work for instance with Views (views_view) suggestions (after #2923634: [PP-1] Use hook_theme_suggestions in views has been applied).

    One hint can be array_slice() returns an array, so we are searching for an array within the suggestions (flat) array. That's always FALSE. If we just want last element of the array, array_pop is a better alternative?

Also I can't comment on the correct solution from a Theme / Twig prospective. I can see from the twig.engine a big chunk of code has been removed about 'theme_hook_suggestions'. #23 reported "Attempts to redetermine the suggestions and print these as the file name suggestions.", but maybe a bit more details?

We do need some test coverage. :(

jwilson3’s picture

Issue tags: +TX (Themer Experience)
markhalliwell’s picture

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mdolnik’s picture

thomas.frobieter’s picture

After patch #33 is applied, all file override suggestions in the twig debugging comments are lost in our case.

anybody’s picture

Additional information and workaround... I'm not sure if this is the same problem / reason, but it seems it could be...

All container suggestions are having the wrong order:

 FILE NAME SUGGESTIONS:
   x container--more-link.html.twig
   * container--more-link--frontpage.html.twig
   * container--more-link--block.html.twig
   * container--more-link--frontpage--block.html.twig
   * container--more-link--block-header.html.twig
   * container--more-link--frontpage--block-header.html.twig
   * container.html.twig

As you can see the more specific suggestions are at the bottom (excluding container.html.twig) while they should be at the top.
Patch #5 didn't solve that for us and I couldn't find the place where these suggestions are built. I was looking for something like "..._suggestions_container()" in core but it seems to be a more general logic?

As quickfix we now used

/**
 * Implements HOOK_theme_suggestions_container_alter
 *
 * @param array $suggestions
 * @param array $variables
 * @return void
 */
function MYTHEME_theme_suggestions_container_alter(array &$suggestions, array $variables)
{
  $suggestions = array_reverse($suggestions);
}

but of course this shouldn't be required and is DANGEROUS if the reason / bug is fixed... because then we'd create the wrong order here.

(So we created a little messy detection:

if(!empty($suggestions)){
    $count = count($suggestions);
    if($count >= 2){
      // Only if there are at least 2 suggestions!
      if(substr_count($suggestions[0], '__') > substr_count($suggestions[$count-1], '__')){
        // Only if the order is possibly wrong:
        $suggestions = array_reverse($suggestions);
      } else {
        // Notify us if the bug is hopefully fixed:
        \Drupal::logger('webksdct')->notice('MYTHEME_theme_suggestions_container_alter: See https://www.drupal.org/project/drupal/issues/2752443#comment-12782864 is hopefully fixed if this log message appears!');
      }
    }
  } 

The result is

 FILE NAME SUGGESTIONS:
   x container--more-link--frontpage--block-header.html.twig
   * container--more-link--block-header.html.twig
   * container--more-link--frontpage--block.html.twig
   * container--more-link--block.html.twig
   * container--more-link--frontpage.html.twig
   * container--more-link.html.twig
   * container.html.twig

which is correct.

Does this issue address this problem? And if yes, why doesn't #5 solve it? (Add a test for that?)

aaronmchale’s picture

I'm guessing this is also what is happening to CSS files where multiple levels of sub-themes/base-themes are used, where some CSS files are loaded in the wrong order breaking some styling.

Order should be:
theme1
theme2
theme3
theme4

Order actually is:
theme1
theme3
theme2
theme4

Update: in the libraries.yml file, if you set the weight of the CSS (e.g. assets/theme.css: { weight: 1 }) it seems to have an effect, I wonder if this is some kind of edge case where the CSS files that are loading in the wrong order are doing that because they have the same name?

Update2: the naming doesn't seem to matter, issue still happens, so essentially a workaround is using weights but since this might not always be practical so this looks like a bug.

markhalliwell’s picture

#47 has nothing to do with this issue.

That being said, the "issue" you're experiencing is due to how Drupal 8 now handles assets (via libraries). All you need to do is add dependencies to your theme libraries. Doing this will automatically ensure they're loaded in the correct order. You don't have to manually fiddle with weights (which can no longer be positive integers anyway).

aaronmchale’s picture

@markcarver thanks for the input, that seems to work, strange that the order isn't correctly simply by setting the base theme

markhalliwell’s picture

Still off-topic, but figured it'd be easier to just respond here.

strange that the order isn't correctly simply by setting the base theme

It cannot and likely never will due to the general complexities of how libraries (and their dependencies) function.

The idea behind libraries is that the developer should know what the libraries intentions are and define them accordingly, manually (once).

How would Drupal know that the 3rd library defined in your theme should be dependent on the 5th library defined in the base-theme?

Is it supposed to just assume (which is never good) that maybe it's perhaps just the first library defined in each that should be linked? That too would need additional documentation as well to handle this "magic" association. What if this dynamic association isn't wanted, one would have to alter the library they just defined to remove it.

Also, simply making all libraries dependent on all the base-theme libraries wouldn't work either. What if one of the libraries is a framework and needs to be loaded before anything else, including a base-theme?

---

I think the confusion around libraries stems from the fact that most simple sites/themes do "operate" in the perceived fashion of "automagic dependencies" due to how Drupal loads things only as they're needed. Assets are generally added in a FIFO order and when the workflow of assets are simplistic in nature like this, that's usually what you get.

That being said, if an extension does any complex library handling, like alters (e.g. a module like advagg or a base-theme that dynamically adds in an external framework like bootstrap), the weight in which these assets are actually loaded can sometimes get skewed.

Libraries are, for the most part, a self-contained system/hierarchy unto themselves. They don't really know the top-level Drupal add/load order, just that they were given an asset without dependencies. So unless you give them proper dependencies, the library asset handler doesn't really have a clue as to how best handle them; it just spits it out when and where ever it can.

The easiest solution to "fix" this is to quite simply always be explicit about which libraries your library depends on. Then there's no confusion, for anyone involved.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

bkosborne’s picture

RE: #47 - #50, someone had opened an issue relating to this and I just cleaned it up: #3070381: Base theme libraries may be output in reverse order. Mark is right though, the "fix" is to just properly understand how the library dependency system works.

luke.stewart’s picture

I'm still seeing the behaviour as originally described on 8.9.0-beta2.
This is for a site using a subtheme of bootstrap_barrio -> bootstrap_sass. The use case is including the same menu twice on the page and attempting to format it differently depending on the block it is in.

The patch applies cleanly to 8.9.0-beta2.
The patch appears to resolve part of the bug in that the order of the template suggestions as output by twig debug appears to be correct but the more specific one is being selected.

Before:

<!-- FILE NAME SUGGESTIONS:
   x menu--main.html.twig
   * menu--main--mainnavigation-2.html.twig
   x menu--main.html.twig
   * menu.html.twig
-->

After:

<!-- FILE NAME SUGGESTIONS:
   * menu--main--mainnavigation-2.html.twig
   x menu--main.html.twig
   * menu.html.twig
-->

I've only noticed this behaviour for menus.
I'm not seeing behaviour described in #45.
The work around suggested in #46 didn't work. menu--main.html.twig still got added at the start.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

joseph.olstad’s picture

patch still applies to 9.0.x - Triggered tests
patch still applies to 9.1.x - Triggered tests
patch still applies to 8.9.x - Triggered tests
patch still applies to 8.8.x - Triggered tests

we still need this, trying to get our patch list down to a more manageable number, would be nice to nail this one, doesn't look like much work is needed to get this in.

see comment #40 for some nit picking todo changes
#2752443-40: Incorrect order and duplicate theme hook suggestions

ofrommel’s picture

I am having the same problem with Drupal 8.9.1 and the Bootstrap Radix theme:

   x node--webinar--full--noaccess.html.twig
   * node--webinar--noaccess.html.twig
   * node--noaccess.html.twig
   * node--webinar--51--noaccess--something--something.html.twig
   x node--webinar--full--noaccess.html.twig
   * node--51--full--noaccess.html.twig
   * node--webinar--noaccess.html.twig
   * node--noaccess.html.twig
   * node--webinar--51--noaccess--something--something.html.twig
   * node--51--full.html.twig
   * node--51.html.twig
   * node--webinar--full.html.twig
   * node--webinar.html.twig
   * node--full.html.twig
   * node.html.twig

You can see I also made up some template suggestions to test specificity but no matter what always the same template gets picked (which is in the list twice). I also tried the patch but it didn't change anything.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

johnalbin’s picture

FYI, the latest MR in #2118743: Twig debug output does not display all suggestions when an array of theme hooks is passed to #theme may fix this issue at the same time.

Actually, looking at this closely, I see that Drupal core's views_theme_suggestions_container_alter() is implemented incorrectly; the return values of buildThemeFunctions() should be reversed before merging them with $suggetions. So that should be fixed in this issue.

joseph.olstad’s picture

@JohnAlbin, thanks for this note, I was struggling with views theme functions and suggestions this week for a custom module, I will have a look at that merge request #2118743 .

johnalbin’s picture

I went ahead and split off the Views more link container bug into its own issue since it is so self-contained. #3188122: Views more link container theme suggestions are in the wrong order

And, yeah, the MR in #2118743: Twig debug output does not display all suggestions when an array of theme hooks is passed to #theme definitely fixes this issue at the same time. Looking at the last patch in this issue, I can see that the approaches are very similar; moving all the code in twig.engine into ThemeManager.php.

anybody’s picture

@JohnAlbin, thank you very much for the progress and impressive work on the other issues. So if the problems documented here are represented in the other issues tests (I didn't have the time to check that yet), I guess we can close this one as duplicate and link them for reference?

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

larowlan’s picture

Issue tags: +Bug Smash Initiative

Came here from #2994975: Base theme hooks are executed in the incorrect order which is similar but different.

  1. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -155,14 +155,19 @@ public function render($hook, array $variables) {
    +    $derived_suggestions[] = $derived_suggestion = $original_hook;
    

    can we split this into multiple assignments, its hard to understand what this is doing and we generally avoid this kind of assignment in core

  2. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -155,14 +155,19 @@ public function render($hook, array $variables) {
    +      foreach (array_slice($derived_suggestions, 1) as $hook) {
    

    it would be good to put this into a variable to convey what it represents, would make it easier to read.

    e.g. $derived_suggestions_without_original_hook

  3. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -228,12 +233,14 @@ public function render($hook, array $variables) {
    +    if (isset($info['base hook']) && !in_array($hook, array_slice($derived_suggestions, 0, -1))) {
    ...
    +    $suggestions = array_merge($suggestions, array_reverse(array_slice($derived_suggestions, 0, -1)));
    

    we call array_slice($derived_suggestions, 0, -1) twice here, for performance but more-so for readability, I think it would be good to assign this to a variable that conveys what this represents.

    e.g. $derived_suggestions_excluding_base_hook

  4. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -244,6 +251,11 @@ public function render($hook, array $variables) {
    +    // Add the base suggestion, should always have the lowest priority
    
    @@ -381,6 +393,8 @@ public function render($hook, array $variables) {
    +      // Make sure we pass the file name suggestions that were actually used
    

    Nit needs a trailing . here

  5. +++ b/core/lib/Drupal/Core/Theme/ThemeManager.php
    @@ -244,6 +251,11 @@ public function render($hook, array $variables) {
    +    if (!in_array(array_slice($derived_suggestions, -1), $suggestions)) {
    

    same here this could be $derived_base_hook, rather than call it twice

ravi.shankar’s picture

StatusFileSize
new3.39 KB
new5.59 KB

Addressed comment #65, please review.

star-szr’s picture

Status: Needs work » Needs review
aaronmchale’s picture

Status: Needs review » Needs work

Patch in #66 failed testing, so status remains at needs work.

star-szr’s picture

I have yet to see any test results, so tried to requeue.

Edit: It's just saying build successful, maybe this is a new thing, I'm used to seeing a pass/fail here in the issue. Clicking through to Jenkins I see that there are failing tests.

joseph.olstad’s picture

re: #66
From the test runner logs:

Undefined variable $derived_suggestion in /var/www/html/core/lib/Drupal/Core/Theme/ThemeManager.php on line 160
Undefined variable $derived_suggestion in /var/www/html/core/lib/Drupal/Core/Theme/ThemeManager.php on line 162
aaronmchale’s picture

Edit: It's just saying build successful, maybe this is a new thing, I'm used to seeing a pass/fail here in the issue. Clicking through to Jenkins I see that there are failing tests.

My understanding is that if there are too many fails it will default to "Build successful".

yogeshmpawar’s picture

Status: Needs work » Needs review
StatusFileSize
new5.58 KB
new744 bytes

Updated patch will fix build/failed test issue

ranjith_kumar_k_u’s picture

StatusFileSize
new5.58 KB
new599 bytes

Fixed CS error

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Anthony Thomas made their first commit to this issue’s fork.

anthony thomas’s picture

StatusFileSize
new5.56 KB

For me, order of template suggestions still wrong.
Edit #73 patch to fix this

$suggestions = array_merge($derived_suggestions_excluding_base_hook, $suggestions);

ravi.shankar’s picture

StatusFileSize
new5.57 KB

Added reroll of patch #76 as it's not getting applied.

Status: Needs review » Needs work

The last submitted patch, 77: 2752443-77.patch, failed testing. View results

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

nicxvan’s picture

I created a related issue since this seems to happen at the page template level too. The patch here does not resolve the page level template issue so it probably makes sense to track separately: https://www.drupal.org/project/drupal/issues/3314943

greggmarshall’s picture

It would appear this patch and the latest one from #2118743: Twig debug output does not display all suggestions when an array of theme hooks is passed to #theme conflict with each other.

ollu’s picture

Patch #73 applies and the output does not contain any duplicates though suggestion added via code does not get picked up by system and by that failing.

FILE NAME SUGGESTIONS:
* input--textfield-site-search.html.twig (*)
* input--textfield.html.twig
x input.html.twig

(*) Added via suggestion hook and is not used despite being present in template folder for the current theme.

ameymudras’s picture

Status: Needs work » Needs review
StatusFileSize
new5.48 KB

#77 wasn't getting applied to 10.1.x, I have re-rolled the above patch. Couldn't generate the interdiff

Status: Needs review » Needs work

The last submitted patch, 83: 2752443-83.patch, failed testing. View results

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

stomusic’s picture

StatusFileSize
new5.92 KB

#77 wasn't getting applied to 10.1.0, #83 also. I've make re-roll

thomas.frobieter’s picture

#86 doesn't work well for me, admin menu bar suggestion comment (as an example, the error appears multiple times for different templates):

INVALID FILE NAME SUGGESTIONS:
   See https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Render!theme.api.php/function/hook_theme_suggestions_alter
   menu__admin
   menu
jacobbell84’s picture

Status: Needs work » Needs review
StatusFileSize
new5.56 KB

I also experienced the same error from this patch. I think the patch has a couple different issues. First, it looks like a lot of the derived suggestion logic was pulled from the twig.engine debug code, but the problem with that is that the debug output is presented in reverse order.

    $derived_suggestion = $original_hook;
    $derived_suggestions[] = $derived_suggestion;
    while ($pos = strrpos($derived_suggestion, '__')) {
      $derived_suggestion = substr($derived_suggestion, 0, $pos);
      $derived_suggestions[] = $derived_suggestion;
    }

This would generate hooks in the order of

base_hook__specific__really_specific
base_hook__specific
base_hook

When the later code merged this array with the suggestions array coming from the theme system (which is in the opposite order) it would create odd sorting under some conditions.

    $derived_suggestions_excluding_base_hook = array_slice($derived_suggestions, 0, -1);
    if (isset($info['base hook']) && !in_array($hook, $derived_suggestions_excluding_base_hook)) {
       $suggestions[] = $hook;
     }
    $suggestions = array_merge($derived_suggestions_excluding_base_hook, $suggestions);

Second, the patch had this check:

    $derived_base_hook = array_slice($derived_suggestions, -1);
    if (!in_array($derived_base_hook, $suggestions)) {
      $suggestions = array_merge($derived_base_hook, $suggestions);
    }

The above code would always fire since array_slice returns an array not a scaler value, which could sometimes cause duplicates.

Third, part of the updates removed some variable initialization in twig.engine

-      
-       // Get the value of the base hook (last derived suggestion) and append it
-      // to the end of all theme suggestions.
-      $base_hook = array_pop($derived_suggestions);

Later in the twig.engine file, the base_hook variable is used to determine whether a suggestion is valid or not

      $base_hook = $base_hook ?? $variables['theme_hook_original'];
      foreach ($suggestions as $key => &$suggestion) {
        // Valid suggestions are $base_hook, $base_hook__*, and contain no hyphens.
        if (($suggestion !== $base_hook && !str_starts_with($suggestion, $base_hook . '__')) || str_contains($suggestion, '-')) {
          $invalid_suggestions[] = $suggestion;
          unset($suggestions[$key]);
          continue;
        }
        $template = strtr($suggestion, '_', '-') . $extension;
        $prefix = ($template == $current_template) ? 'x' : '*';
        $suggestion = $prefix . ' ' . $template;
      }

That's the reason folks started getting the "INVALID FILE NAME SUGGESTIONS:" error after applying the patch. I made a new patch that addresses those 3 issues.

smustgrave’s picture

Status: Needs review » Needs work

Seems to have failures and is missing test coverage. Moving to NW for that.

jacobbell84’s picture

StatusFileSize
new5.56 KB

Attempting to fix the failures. I assume we'll want to wait until it's confirmed this is the approach people want before we write test cases though.

jacobbell84’s picture

StatusFileSize
new5.55 KB

Fixing failing unit test.

joseph.olstad’s picture

Status: Needs work » Needs review
joseph.olstad’s picture

Status: Needs review » Needs work

Actually, still needs tests.

jacobbell84’s picture

Yup, I just wanted to fix the issues first. I'll try to get a unit test going, but I have very little experience with it so it might take a bit.

nicxvan’s picture

This should probably move to an MR, the core team is using MRs for development now and the test suite is significantly faster on gitlab if you're writing tests.

nicxvan’s picture

I've converted the patch in 91 as closely as I can to 11.x, there was a refactor in https://www.drupal.org/project/drupal/issues/2953921 that may have affected how this works. I have not added any additional tests, but I will review the MR now to see if it is working.

Hiding patches.

nicxvan’s picture

nicxvan’s picture

Can someone confirm this is still an issue on 11.x

I was unable to reproduce this, if you can update the steps to reproduce I'm happy to continue testing.

joseph.olstad’s picture

@nicxvan, the WxT distribution has been using patch #1 for several years.

With that said, we're using the bootstrap 3x theme which may not behave exactly as the core themes. It would be good to try the latest version of this patch and give it some serious looks.

Unfortunately I don't have any spare cycles at this moment.

nicxvan’s picture

There was some refactoring in the same area so I think that might have resolved this issue, but I'd like another confirmation.

ayalon’s picture

StatusFileSize
new7.03 KB

Thanks for the updated patch. Here is an updated patch file for use with Drupal 10.2 based on the merge request usable with composer.json

joseph.olstad’s picture

phplint saw an undefined variable, replaced with what looked to be obviously the correct variable.

alphex’s picture

I can confirm this is still happening with Drupal 10.2.5 - WITH - the patch from #102 applied.

<!-- FILE NAME SUGGESTIONS:
   x page--node--general-page.html.twig
   * page--node--2.html.twig
   * page--node--%.html.twig
   * page--node.html.twig
   * page.html.twig
-->

I would expect that "page--node--2.html.twig" would be at the top of the list.

----

When I add some code I've used in past sites, to my .theme file

function themename_theme_suggestions_page_alter(array &$suggestions, array $variables) {
  // Add content type suggestions.
  if ($node = \Drupal::request()->attributes->get('node')) {
    array_splice($suggestions, 1, 0, 'page__node__' . $node->getType());
  }
}

It puts the template suggestion, in the right spot, but the TOP one is still there, which is wrong.

<!-- FILE NAME SUGGESTIONS:
   x page--node--general-page.html.twig
   * page--node--2.html.twig
   * page--node--%.html.twig
   x page--node--general-page.html.twig
   * page--node.html.twig
   * page.html.twig
-->
sapetm’s picture

StatusFileSize
new4.51 KB

Sorry, bad patch file, please disregard.

sapetm’s picture

StatusFileSize
new4.51 KB

Sorry, bad patch file, please disregard.

sapetm’s picture

StatusFileSize
new7.04 KB

Sorry folks, third time's the charm, or something like that. I uploaded the wrong patch file with my other two comments. This is the patch I am using for Drupal 10.2.6 and PHP 8.1.27. It is identical to the patch in comment #102 but with the addition of an array_unique() in the second hunk when setting the value of the $suggestions array in the render function. This fixes the issue for us. Hope it helps!

nicxvan’s picture

Can you please add those changes to the merge request? People can then generate local patches as needed and the core team uses the actual mr for testing and merging.

scott_euser’s picture

StatusFileSize
new22.23 KB

Looks like the issue summary needs an update? Drupal 11 with default Olivero theme, no changes from clean install other than turning on twig.config debug to true now results in this which appears correct:

<!-- FILE NAME SUGGESTIONS:
   ▪️ menu--main.html.twig
   ✅ menu--primary-menu.html.twig
   ▪️ menu.html.twig
-->

The selected customisation comes from olivero.theme:

function olivero_theme_suggestions_menu_alter(&$suggestions, array $variables) {
  if (isset($variables['attributes']['region'])) {
    $suggestions[] = 'menu__' . $variables['attributes']['region'];
  }
}

For the region 'primary_menu'. Is there a step I am missing to reproduce?

djroshi’s picture

I'm going to post this as it might help someone who stumbles over this issue (as I did). This is using Drupal 10.2.6 with Bootstrap5 subtheme.

When placing a menu block in a region (pretty normal thing to do) I get the following suggestions, prioritized first to last (as suggestions are array_reversed)

1. menu__region-name
2. menu__menu-name_region-name
3. menu__menu-name

This is counterintuitive because (in my mind at least) theme suggestions should be ordered from most specific to least specific i.e.

1. menu__menu-name_region-name (that menu when it appears in that region)
2. menu__menu-name (all instances of that menu)
3. menu__region-name (all menus in that region)

Can someone perhaps confirm if this is happening on other installs of Drupal? Or is it just me :D

anybody’s picture

Re #110 you're right and that's what this issue is about.

We're still experiencing this with Drupal 10.3.0, so I don't think this is fixed without this patch.

Update: re-tried the MR and it seems it works correctly for us.
Duplicate entries are gone and the Twig Debug Theme suggestions output is now ordered correctly! Drupal 10.3.0

anybody’s picture

dinesh18’s picture

Is there a fix for this?

welly’s picture

I've tested the patch at #107 on Drupal 10.4.5 and confirm this works. It fixed an issue I was having that was exactly as described. This was after an upgrade of group_content_menu from 3.0.1 to 3.0.5 and Drupal 10.2.12 to 10.4.5.

xjm credited alexpott.

xjm credited joelpittet.

xjm’s picture

Per @lauriii in #30, this was triaged by

Cottser, alexpott, xjm, joelpittet

Adding triage credits.

voleger’s picture

Added link to #1685492: Convert theme engines into services as the theme engine location was changed.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.