Follow-up from #591794: Allow themes to alter forms and page. In that issue we made it so that themes can implement alter hooks. We documented this feature and addressed the performance concerns through static caching within drupal_alter(). We agreed that it's okay for themes to not have a chance to participate in alter hooks that run during bootstrap (e.g., hook_query_alter(), hook_url_inbound_alter()), but that the whole point of the feature is for them to participate in hook_form_alter(), hook_page_alter(), and similar hooks that relate to rendering.
When the feature was first added to D7, not all HEAD code was converted to using the new render api correctly, so theme() was likely to be called somewhat early during page callback execution, making it so that the theme was loaded and able to participate in the alter hooks. Now, theme() should not be called until after drupal_alter('page'), as page callback execution should only return render arrays, and all rendering should happen after the page has been altered. So now, themes aren't able to participate in the alter hooks. Or rather, whether they are or not depends on odd factors. For example, if a form does not set #theme, then drupal_build_form() calls drupal_theme_initialize(): too late for the theme to participate in hook_form_alter(), but early enough for it to participate in hook_page_alter(). But, if the form does set #theme, then drupal_theme_initialize() is not called, and the theme isn't able to participate in hook_page_alter() unless some other random code happened to call drupal_theme_initialize().
IMO, this inconsistency of whether a theme's alter hook implementation runs or not, depending on tough to trace down factors, is a critical bug. I think prior to D7 release, we must either roll back the feature entirely, or ensure that the theme is loaded early enough for it to participate consistently. #591794-94: Allow themes to alter forms and page contains a patch for the latter, but Moshe suggests that to do something along those lines, we need to separate the loading of the theme from initializing its registry, because if the registry isn't cached already, then building it takes a long time, and it would be a shame to force AJAX callbacks or other requests that don't require theming to incur that cost.
Comment | File | Size | Author |
---|---|---|---|
#36 | 812016-theme-init-early-36.patch | 10.92 KB | effulgentsia |
#34 | 812016-theme-init-early-34.patch | 8.39 KB | effulgentsia |
#30 | 812016-theme-init-early-2.patch | 7.95 KB | redndahead |
#29 | 812016-theme-init-early-1.patch | 7.95 KB | redndahead |
#26 | 812016-theme-init-early.patch | 7.67 KB | Damien Tournoud |
Comments
Comment #1
sun1) We need actual data for when/how often/in what requests themes are not initialized at all currently, so as to be able to evaluate the performance impact of always initializing themes early.
2) I originally stated the idea of loading vs. initializing, but I'm actually not sure whether that's possible or whether there's a real difference at all? If this is possible, then it sounds like the right solution attempt to me, since just loading 1-2 files more (template.php + base theme) in every request shouldn't have a large performance impact.
3) I wouldn't support rolling back the entire feature. Sounds like turning around after walking to 99%.
Either way, this will likely result in an API change.
Comment #2
Nick Lewis CreditAttribution: Nick Lewis commentedOye. So basically we have a phantom alter API in the theme. Even worse, its an unpredictable phantom of mysterious habits.
How many strange alter functions are there between bootstrap and return $output;? Would it maybe make sense to start by verifying what's supposed to run figuring out test cases for them?
So... http://api.drupal.org/api/function/drupal_theme_initialize/7 just spotchecking this function i gotta admit that i feel like something smells fishy. Why is it being called in so many places? Would it be to safe to say that finding it in places like the l() function is related to this issue?
Comment #3
Nick Lewis CreditAttribution: Nick Lewis commentedSorry about the glitch sun (i didn't change that title lol) And you'll have to add back the API change tag - my comment totally ate it.
Comment #4
effulgentsia CreditAttribution: effulgentsia commentedIt is called wherever code needs to know if a particular theme hook is implemented by the theme. Used for dynamically deciding whether to call theme(FOO) or set #theme=FOO based on whether the theme actually implements THEMENAME_FOO() or FOO.tpl.php. The theme must be loaded and the registry retrieved (or built) to ask that question. An API function to ask the question (instead of initialize/get_registry) would be nice, perhaps something for D8. The fact that it's called from many places, including l(), increases the odds of the theme participating in alter hooks that happen later in the page request.
[EDIT: small correction: the test of whether a theme hook is in the registry is not about whether a particular theme overrides a default implementation, but whether the hook exists at all. For example, if $form_id = 'article_node_form', the theme hook can exist in the registry if there's a theme_article_node_form() or a THEMENAME_article_node_form() or a article-node-form.tpl.php file in the theme or a base theme or registered by a module. Anyway, a minor distinction from the perspective of the need of code for sometimes to know if a dynamic string has a corresponding theme hook, but didn't want to leave the comment with misleading info.]
Comment #5
amitaibusubscribe.
Comment #6
rfaysubscribe
Comment #7
effulgentsia CreditAttribution: effulgentsia commentedRe #1.1:
So the question is which page requests can complete without ever loading the theme. Well, some AJAX requests (for example, autocomplete), but not all AJAX requests ("add another item" returns themed HTML). While not in core, we should consider services and drush. Also form submissions (the POST request doesn't require rendering) and anything else that ends in a drupal_goto().
I think the example to use for analyzing performance is "user/autocomplete/r". On a clean drupal install if the uid=1 user is "root", this just returns
{"root":"root"}
. This can be benchmarked using Apache Benchmark ("ab") as long as the anonymous user has "view user profiles" permission.1. HEAD: 19.4ms
2. Patch from #591794-94: Allow themes to alter forms and page: 22.1ms (once theme registry is cached)
3. Patch + modification of _theme_load_registry() to comment out calling _theme_save_registry(): 97ms (so yes, building the registry is expensive, but i'm not really sure when that matters, as once it's built once, it's then cached)
4. Patch + modification of _theme_load_registry() to add a
return;
as the first line: 19.7ms (testing all of drupal_theme_initialize() minus getting the registry from cache)Based on the above, I think it makes sense to add an optional $load_registry parameter that defaults to TRUE to drupal_theme_initialize(). In and of itself, I don't think that's an API change (just an API addition): the question is whether the implementation can be done in a way that doesn't have other side effects and that doesn't involve a collateral API change.
Note: some may argue that even the 0.3ms overhead of #4 is too much. I don't think it is when it takes 20ms just to bootstrap. But we might want to consider things like http://drupal.org/project/js. Though a contrib project that adds a replacement to index.php can also add a replacement to menu_execute_active_handler(), so I don't know if we really need to take that into account.
Comment #8
moshe weitzman CreditAttribution: moshe weitzman commentedYeah, I agree with adding a $load_registry param and with you swatting away any collateral damage :). I actually think there won't be any damage. Seems like you just have take your patch in #591794-94: Allow themes to alter forms and page and pass FALSE in both calls to drupal_theme_initialize().
Based on the above data, it takes 2.4ms just to retrieve and unserialize the theme registry (22.1-19.7). Thats a problem that gets much worse as you add more modules (i.e. more theme hooks). Lets see what we can do here for D8.
-moshe
Comment #9
effulgentsia CreditAttribution: effulgentsia commentedMight be useful to add a test to the patch, but I think this is all that's needed otherwise. Once we're in drupal_render_page(), we're very likely going to call theme() at least once, so I don't think that one needs to pass FALSE.
Comment #10
moshe weitzman CreditAttribution: moshe weitzman commentedThis is perfect, IMO. Please wait for green before commit.
Comment #11
Nick Lewis CreditAttribution: Nick Lewis commentedI concur with Moshe. And its green btw.
Comment #12
YesCT CreditAttribution: YesCT commented#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #14
catch#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #16
catch#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #17
aspilicious CreditAttribution: aspilicious commentedBack to rtbc by #10 and #11
Comment #18
bjalford CreditAttribution: bjalford commented#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #19
YesCT CreditAttribution: YesCT commented#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #20
marcingy CreditAttribution: marcingy commented#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #21
bleen CreditAttribution: bleen commented#9 = green ... #10 = RTBC (moshe) ... #20 = still green
Comment #22
marcingy CreditAttribution: marcingy commented#9: theme-intialize-early-812016-9.patch queued for re-testing.
Comment #23
webchickThanks for the nice issue summary, and for figuring out a way to fix it without an API change. :)
We need to document this new parameter.
And I really, really would love a test for this. Maybe hitting an autocomplete URL and ensuring that the theme system hasn't fully initialized, somehow?
Powered by Dreditor.
Comment #24
chx CreditAttribution: chx commentedYour wish is my command.
Comment #25
chx CreditAttribution: chx commentedOh. You wanted a comment too. Sure.
Comment #26
Damien Tournoud CreditAttribution: Damien Tournoud commentedWhat about something like this instead?
I don't understand why we have a push system here, instead of a pull one. This whole area should be refactored, but this is a minimal fix.
Comment #27
chx CreditAttribution: chx commentedThat's a tricky way to do it but no public API changed at all so that's even better! Private functions can change at will, no probs.
To recap, Damien's patch stores the registry callback in
theme_get_registry
and makes it the registry load on demand. So it's only actually callingtheme_get_registry
that loads the registry and that happens on the firsttheme()
call. Previously, on theme initialize we fetched the registry and pushed it intotheme_get_registry
. Edit: so instead of pushing the whole registry we only push the registry callback and because of that a private API change happens, the registry pusher is gone and we introduce a registry callback pusher. All these pushers, "I got things you won't believe" :pComment #28
webchickI think this makes sense. And I guess if there are any fall-out from moving theme initialization earlier, testbot would catch it.
However, this no longer applies. Can I get a re-roll, please?
Comment #29
redndahead CreditAttribution: redndahead commentedHere is a re-roll.
Comment #30
redndahead CreditAttribution: redndahead commentedDoes this line need to be array('url' => 'user') it seems that I may have missed that change. Here is the changed patch just in case.
Powered by Dreditor.
Comment #31
JacineAlthough I can't properly review the patch, I'm so relieved to find this issue and just want to say THANK YOU to everyone working on it! I was going insane trying to figure out the proper way to change the formatter of a taxonomy term field (from link to plain text), thinking this hook_field_display_alter() was one of the alter hooks that wasn't permitted for themes. Phew. :)
Comment #32
JacineWith this patch applied, update.php uses the default theme instead of the admin theme like it's supposed to.
Comment #33
David_Rothstein CreditAttribution: David_Rothstein commented@webkenny, I, and some others just looked into this a bit.
It looks like update.php does this:
and then drupal_maintenance_theme() refuses to set the theme to the maintenance theme because with the current patch the theme has already been set in the bootstrap - that is probably the cause of the bug Jacine found.
This is sort of due to drupal_maintenance_theme() using global variables to set the theme (rather than the new methods in D7), but given how much of a special case maintenance pages are that might not be easy to fix....
Another option would be to change how the drupal_theme_initialize() call is added by this patch. Currently the bootstrap does this:
The patch here adds drupal_theme_initialize() after the menu_set_custom_theme() call but outside the if() statement. Maybe it should be in the if() statement as well, for the same reason the module_invoke_all('init') call is? Not sure if there are any bad side effects to that.
Either way, I also noticed that the code comment above (the part about "... since any modules which initialize the theme system...") is out of date as a result of this patch, since we aren't relying on modules to initialize the theme system but rather are now explicitly doing it ourselves.
Comment #34
effulgentsia CreditAttribution: effulgentsia commentedWith the change suggested by #33. The rest of this looks RTBC to me, but setting to "needs review" so the change can be reviewed.
Comment #35
JacineI just (happily) tested the patch in #34 with a few themes and all appears to be working well! :)
Comment #36
effulgentsia CreditAttribution: effulgentsia commentedThanks, Jacine. This adds a test for the initial issue (and I confirmed that the test without the rest of the patch fails in HEAD), so now we have a test that the theme alter hooks run, and that the registry is not loaded for user/*/autocomplete. IMHO, this is ready.
Comment #37
sunyay, thanks!
For D8, we should make the maintenance mode context set a custom theme, just like any other context.
Comment #38
David_Rothstein CreditAttribution: David_Rothstein commentedLooks good - a minor point is that menu_set_custom_theme() already has an internal check for MAINTENANCE_MODE, so putting menu_set_custom_theme() inside the if() statement with the rest of them isn't necessary, and just leads to a duplicate check.
However, for the purposes of code readability, it is probably reasonable to move it, since the other theme stuff has to be there.
Comment #39
Dries CreditAttribution: Dries commentedAlright. Committed to CVS HEAD.