Follow-up from #591794: Allow themes to alter forms and page. In that issue we made it so that themes can implement alter hooks. We documented this feature and addressed the performance concerns through static caching within drupal_alter(). We agreed that it's okay for themes to not have a chance to participate in alter hooks that run during bootstrap (e.g., hook_query_alter(), hook_url_inbound_alter()), but that the whole point of the feature is for them to participate in hook_form_alter(), hook_page_alter(), and similar hooks that relate to rendering.

When the feature was first added to D7, not all HEAD code was converted to using the new render api correctly, so theme() was likely to be called somewhat early during page callback execution, making it so that the theme was loaded and able to participate in the alter hooks. Now, theme() should not be called until after drupal_alter('page'), as page callback execution should only return render arrays, and all rendering should happen after the page has been altered. So now, themes aren't able to participate in the alter hooks. Or rather, whether they are or not depends on odd factors. For example, if a form does not set #theme, then drupal_build_form() calls drupal_theme_initialize(): too late for the theme to participate in hook_form_alter(), but early enough for it to participate in hook_page_alter(). But, if the form does set #theme, then drupal_theme_initialize() is not called, and the theme isn't able to participate in hook_page_alter() unless some other random code happened to call drupal_theme_initialize().

IMO, this inconsistency of whether a theme's alter hook implementation runs or not, depending on tough to trace down factors, is a critical bug. I think prior to D7 release, we must either roll back the feature entirely, or ensure that the theme is loaded early enough for it to participate consistently. #591794-94: Allow themes to alter forms and page contains a patch for the latter, but Moshe suggests that to do something along those lines, we need to separate the loading of the theme from initializing its registry, because if the registry isn't cached already, then building it takes a long time, and it would be a shame to force AJAX callbacks or other requests that don't require theming to incur that cost.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sun’s picture

Title: Whether a theme can participate during drupal_alter() depends on whether drupal_theme_initialize() happened to be called » Themes cannot always participate in drupal_alter()
Issue tags: +API change

1) We need actual data for when/how often/in what requests themes are not initialized at all currently, so as to be able to evaluate the performance impact of always initializing themes early.

2) I originally stated the idea of loading vs. initializing, but I'm actually not sure whether that's possible or whether there's a real difference at all? If this is possible, then it sounds like the right solution attempt to me, since just loading 1-2 files more (template.php + base theme) in every request shouldn't have a large performance impact.

3) I wouldn't support rolling back the entire feature. Sounds like turning around after walking to 99%.

Either way, this will likely result in an API change.

Nick Lewis’s picture

Title: Themes cannot always participate in drupal_alter() » Whether a theme can participate during drupal_alter() depends on whether drupal_theme_initialize() happened to be called
Issue tags: -API change

Oye. So basically we have a phantom alter API in the theme. Even worse, its an unpredictable phantom of mysterious habits.

How many strange alter functions are there between bootstrap and return $output;? Would it maybe make sense to start by verifying what's supposed to run figuring out test cases for them?

So... http://api.drupal.org/api/function/drupal_theme_initialize/7 just spotchecking this function i gotta admit that i feel like something smells fishy. Why is it being called in so many places? Would it be to safe to say that finding it in places like the l() function is related to this issue?

Nick Lewis’s picture

Title: Whether a theme can participate during drupal_alter() depends on whether drupal_theme_initialize() happened to be called » Themes cannot always participate in drupal_alter()

Sorry about the glitch sun (i didn't change that title lol) And you'll have to add back the API change tag - my comment totally ate it.

effulgentsia’s picture

Why is it [drupal_theme_initialize()] being called in so many places? Would it be to safe to say that finding it in places like the l() function is related to this issue?

It is called wherever code needs to know if a particular theme hook is implemented by the theme. Used for dynamically deciding whether to call theme(FOO) or set #theme=FOO based on whether the theme actually implements THEMENAME_FOO() or FOO.tpl.php. The theme must be loaded and the registry retrieved (or built) to ask that question. An API function to ask the question (instead of initialize/get_registry) would be nice, perhaps something for D8. The fact that it's called from many places, including l(), increases the odds of the theme participating in alter hooks that happen later in the page request.

[EDIT: small correction: the test of whether a theme hook is in the registry is not about whether a particular theme overrides a default implementation, but whether the hook exists at all. For example, if $form_id = 'article_node_form', the theme hook can exist in the registry if there's a theme_article_node_form() or a THEMENAME_article_node_form() or a article-node-form.tpl.php file in the theme or a base theme or registered by a module. Anyway, a minor distinction from the perspective of the need of code for sometimes to know if a dynamic string has a corresponding theme hook, but didn't want to leave the comment with misleading info.]

amitaibu’s picture

subscribe.

rfay’s picture

subscribe

effulgentsia’s picture

Re #1.1:
So the question is which page requests can complete without ever loading the theme. Well, some AJAX requests (for example, autocomplete), but not all AJAX requests ("add another item" returns themed HTML). While not in core, we should consider services and drush. Also form submissions (the POST request doesn't require rendering) and anything else that ends in a drupal_goto().

I think the example to use for analyzing performance is "user/autocomplete/r". On a clean drupal install if the uid=1 user is "root", this just returns {"root":"root"}. This can be benchmarked using Apache Benchmark ("ab") as long as the anonymous user has "view user profiles" permission.

1. HEAD: 19.4ms
2. Patch from #591794-94: Allow themes to alter forms and page: 22.1ms (once theme registry is cached)
3. Patch + modification of _theme_load_registry() to comment out calling _theme_save_registry(): 97ms (so yes, building the registry is expensive, but i'm not really sure when that matters, as once it's built once, it's then cached)
4. Patch + modification of _theme_load_registry() to add a return; as the first line: 19.7ms (testing all of drupal_theme_initialize() minus getting the registry from cache)

Based on the above, I think it makes sense to add an optional $load_registry parameter that defaults to TRUE to drupal_theme_initialize(). In and of itself, I don't think that's an API change (just an API addition): the question is whether the implementation can be done in a way that doesn't have other side effects and that doesn't involve a collateral API change.

Note: some may argue that even the 0.3ms overhead of #4 is too much. I don't think it is when it takes 20ms just to bootstrap. But we might want to consider things like http://drupal.org/project/js. Though a contrib project that adds a replacement to index.php can also add a replacement to menu_execute_active_handler(), so I don't know if we really need to take that into account.

moshe weitzman’s picture

Yeah, I agree with adding a $load_registry param and with you swatting away any collateral damage :). I actually think there won't be any damage. Seems like you just have take your patch in #591794-94: Allow themes to alter forms and page and pass FALSE in both calls to drupal_theme_initialize().

Based on the above data, it takes 2.4ms just to retrieve and unserialize the theme registry (22.1-19.7). Thats a problem that gets much worse as you add more modules (i.e. more theme hooks). Lets see what we can do here for D8.

-moshe

effulgentsia’s picture

Status: Active » Needs review
FileSize
3.88 KB

Might be useful to add a test to the patch, but I think this is all that's needed otherwise. Once we're in drupal_render_page(), we're very likely going to call theme() at least once, so I don't think that one needs to pass FALSE.

moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

This is perfect, IMO. Please wait for green before commit.

Nick Lewis’s picture

I concur with Moshe. And its green btw.

YesCT’s picture

Status: Reviewed & tested by the community » Needs work

The last submitted patch, theme-intialize-early-812016-9.patch, failed testing.

catch’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, theme-intialize-early-812016-9.patch, failed testing.

catch’s picture

Status: Needs work » Needs review
aspilicious’s picture

Status: Needs review » Reviewed & tested by the community

Back to rtbc by #10 and #11

bjalford’s picture

YesCT’s picture

marcingy’s picture

bleen’s picture

#9 = green ... #10 = RTBC (moshe) ... #20 = still green

marcingy’s picture

webchick’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests

Thanks for the nice issue summary, and for figuring out a way to fix it without an API change. :)

+++ includes/theme.inc	29 May 2010 02:04:59 -0000
@@ -68,11 +68,20 @@ function _drupal_theme_access($theme) {
+function drupal_theme_initialize($load_registry = TRUE) {

We need to document this new parameter.

And I really, really would love a test for this. Maybe hitting an autocomplete URL and ensuring that the theme system hasn't fully initialized, somehow?

Powered by Dreditor.

chx’s picture

Status: Needs work » Needs review
FileSize
6.37 KB

Your wish is my command.

chx’s picture

FileSize
6.86 KB

Oh. You wanted a comment too. Sure.

Damien Tournoud’s picture

What about something like this instead?

I don't understand why we have a push system here, instead of a pull one. This whole area should be refactored, but this is a minimal fix.

chx’s picture

Status: Needs review » Reviewed & tested by the community

That's a tricky way to do it but no public API changed at all so that's even better! Private functions can change at will, no probs.

To recap, Damien's patch stores the registry callback in theme_get_registry and makes it the registry load on demand. So it's only actually calling theme_get_registry that loads the registry and that happens on the first theme() call. Previously, on theme initialize we fetched the registry and pushed it into theme_get_registry. Edit: so instead of pushing the whole registry we only push the registry callback and because of that a private API change happens, the registry pusher is gone and we introduce a registry callback pusher. All these pushers, "I got things you won't believe" :p

webchick’s picture

Status: Reviewed & tested by the community » Needs work

I think this makes sense. And I guess if there are any fall-out from moving theme initialization earlier, testbot would catch it.

However, this no longer applies. Can I get a re-roll, please?

redndahead’s picture

Status: Needs work » Needs review
FileSize
7.95 KB

Here is a re-roll.

redndahead’s picture

+++ modules/simpletest/tests/theme_test.module	5 Aug 2010 14:42:02 -0000
@@ -24,15 +24,36 @@ function theme_test_menu() {
+    $GLOBALS['theme_test_output'] = theme('more_link', array('url' => url('user'), 'title' => 'Themed output generated in hook_init()'));

Does this line need to be array('url' => 'user') it seems that I may have missed that change. Here is the changed patch just in case.

Powered by Dreditor.

Jacine’s picture

Although I can't properly review the patch, I'm so relieved to find this issue and just want to say THANK YOU to everyone working on it! I was going insane trying to figure out the proper way to change the formatter of a taxonomy term field (from link to plain text), thinking this hook_field_display_alter() was one of the alter hooks that wasn't permitted for themes. Phew. :)

Jacine’s picture

Status: Needs review » Needs work

With this patch applied, update.php uses the default theme instead of the admin theme like it's supposed to.

David_Rothstein’s picture

@webkenny, I, and some others just looked into this a bit.

It looks like update.php does this:

drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
drupal_maintenance_theme();

and then drupal_maintenance_theme() refuses to set the theme to the maintenance theme because with the current patch the theme has already been set in the bootstrap - that is probably the cause of the bug Jacine found.

This is sort of due to drupal_maintenance_theme() using global variables to set the theme (rather than the new methods in D7), but given how much of a special case maintenance pages are that might not be easy to fix....

Another option would be to change how the drupal_theme_initialize() call is added by this patch. Currently the bootstrap does this:

  // Set a custom theme for the current page, if there is one. We need to run
  // this before invoking hook_init(), since any modules which initialize the
  // theme system will prevent a custom theme from being correctly set later.
  menu_set_custom_theme();
  // Let all modules take action before menu system handles the request
  // We do not want this while running update.php.
  if (!defined('MAINTENANCE_MODE') || MAINTENANCE_MODE != 'update') {
    module_invoke_all('init');
  }

The patch here adds drupal_theme_initialize() after the menu_set_custom_theme() call but outside the if() statement. Maybe it should be in the if() statement as well, for the same reason the module_invoke_all('init') call is? Not sure if there are any bad side effects to that.

Either way, I also noticed that the code comment above (the part about "... since any modules which initialize the theme system...") is out of date as a result of this patch, since we aren't relying on modules to initialize the theme system but rather are now explicitly doing it ourselves.

effulgentsia’s picture

Status: Needs work » Needs review
FileSize
8.39 KB

With the change suggested by #33. The rest of this looks RTBC to me, but setting to "needs review" so the change can be reviewed.

Jacine’s picture

Issue tags: +Needs tests

I just (happily) tested the patch in #34 with a few themes and all appears to be working well! :)

effulgentsia’s picture

Issue tags: -Needs tests
FileSize
10.92 KB

Thanks, Jacine. This adds a test for the initial issue (and I confirmed that the test without the rest of the patch fails in HEAD), so now we have a test that the theme alter hooks run, and that the registry is not loaded for user/*/autocomplete. IMHO, this is ready.

sun’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

yay, thanks!

For D8, we should make the maintenance mode context set a custom theme, just like any other context.

David_Rothstein’s picture

Looks good - a minor point is that menu_set_custom_theme() already has an internal check for MAINTENANCE_MODE, so putting menu_set_custom_theme() inside the if() statement with the rest of them isn't necessary, and just leads to a duplicate check.

However, for the purposes of code readability, it is probably reasonable to move it, since the other theme stuff has to be there.

Dries’s picture

Status: Reviewed & tested by the community » Fixed

Alright. Committed to CVS HEAD.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.