Problem/Motivation
Currently '#type' can be used to merge in some default renderable array properties http://drupal.org/node/930760:
When #type is declared for a render array, the default properties of the element type named are loaded through the use of hook_element_info().
Separately in drupal_render() we can declare '#theme' which defines the theme function responsible for rendering this array.
It's a very common pattern that in the defaults set by #type, either a #theme or #theme_wrappers attribute is set for the element. So common, in fact, that many people in the community are confused as to what the difference between #type and #theme actually is, and when to use each (at least, outside the context of the FAPI where everything is based on #type).
We have a DX/learnability problem where the expected behaviour of renderable arrays is split across the combination of '#type' and '#theme' or just '#markup' and there's no central documentation for what HTML elements "should" be using #type as the basis for a new render array of #theme. The FAPI docs are amazing, but they still fall short of covering a huge number of renderables in Core.
In short, predicting the behaviour of a given render array can be very complex - easy to learn (sort of) but hard to master. Creating a new render array from scratch and being confident that you've built it "the right way" can take a lot of research and experience.
Moshe said this:
#theme is responsible for turning the array into HTML
#type is used to merge in defaults from hook_element_info()
I believe that while this is true and simple enough in theory, it can be confusing to work with in practice given the current set of defined #type, #theme and #theme_wrappers elements used in core and contrib.
Proposed resolution
I think we could take the idea of #type setting defaults a step further and use it *as* a default theme hook suggestion.
- Using #type to set a default #theme hook is already a very common pattern in the Render API
- If you use the array syntax for theme hooks #2923506: Deprecate and discourage use of the "array of theme hooks" featurearray('hookname')
then it is harmless if theme() does not find a match for the suggestion
- Learning and using render arrays would be easier because you could almost always assume that setting the #type will set the right #theme hook for you automatically and almost never need to worry about #theme again, except when you're overriding the default
- If you wish to override a non-theme based render (#pre_render -> #markup for example), all you have to do is define the theme hook and template, no need to mess with hook_element_info_alter()
There is a slight issue of how to handle #theme_wrappers. For example, consider #type 'radios' that sets #theme_wrappers => array('radios') already as its default - in this case we don't want to add #theme 'radios' as this would lead to theme('radios') being called twice, inappropriately.
We also don't want to touch a render element that has an empty array set for #theme_wrappers, this is because currently core sets an empty array for #theme_wrappers when it wants to use the defaults provided by a #type sans #theme and #theme_wrappers. See form_process_vertical_tabs() for an example of this where vertical tabs use #type 'details' but forcing the call to theme() to be bypassed while still inheriting the #pre_render calls for 'details'.
This means that as well as avoiding adding #type as a default suggestion for #theme when #theme is already set, we also want to avoid using #type as a default #theme if #theme_wrappers is set and empty OR #theme_wrappers is set and already contains #type as a key/value (we need to check the key and value due to the two different syntaxes for #theme_wrappers).
Remaining tasks
There is a patch in #76 to be applied.
User interface changes
None.
API changes
$elements['#type'] will be used as a #theme suggestion in array syntax, ie. $elements['#theme'] = array($elements['#type']) when neither #theme nor #theme_wrappers has already been defined for $elements.
This API change does not break backwards compatibility with existing render arrays as the syntax of the theme suggestion means that theme() and drupal_render() simply ignore the suggested #type-hook if it is not implemented.
Related Issues
This issue is part of #2015147: [meta] Improve the DX of drupal_render(), renderable arrays and the Render API in general.
Implementing the suggested resolution of this issue will make #2025629: [meta] Ensure that all #theme/#theme_wrappers hooks provided by core are associated with a sensible, re-usable element #type much simpler, easier and cleaner - the changes proposed there will be more palatable and be less of an "API breakage" with this patch in place.
Comment | File | Size | Author |
---|---|---|---|
#14 | 2005970-14.patch | 4.44 KB | thedavidmeister |
#19 | 2005970-19.patch | 4.15 KB | thedavidmeister |
#23 | 2005970-23.patch | 15.25 KB | thedavidmeister |
#26 | 2005970-26.patch | 15.25 KB | thedavidmeister |
#36 | 2005970-36.patch | 14.54 KB | thedavidmeister |
Comments
Comment #0.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.2
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.3
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.4
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.5
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.6
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.7
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.8
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.9
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #0.10
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #1
c4rl CreditAttribution: c4rl commentedGreat start, thedavidmeister.
Definitely, this is one of the core principles of the refactoring. #type and #theme currently have convolved responsibility and that needs to change.
The way that hook_element_info() currently works is there's an implicit association here http://drupal.org/node/169815. If a theme callback named identical to the #type it will be used. So this doesn't seem necessarily new.
What would be examples where #type and #theme do not have a 1-to-1 association? A question I have is if we envision a world where #theme goes away entirely? (I think it might). Do we feel that hook_element_info() and hook_theme() are substantively different in purpose to remain separate callbacks?
Comment #2
tim.plunkettAdding something like
would preserve the simplicity of #markup.
Otherwise I'm generally +1 to this.
Comment #3
thedavidmeister CreditAttribution: thedavidmeister commented#1 - Ah yeah, Fabianx and I were having some chats in IRC about possible futures that may or may not happen. I didn't want to preclude them here.
http://drupal.org/node/169815 says:
I thought it important to mention that '#theme' is always assumed to have some value but if it is set to FALSE you can explicitly bypass theme(). This is not how drupal_render() currently works as it only checks isset() for '#theme'.
For example, currently theme() has preprocess and process phases, for which there's a proposal to refactor a bit here #2004872: [meta] Theme system architecture changes. Fabianx explained to me though that *anything* that can be teased out into discrete "input -> processed variables -> altered + processed variables -> string" can be "drillable" by Twig if that workflow is structured so that Twig can invoke the processing without the string concatenation at the end. I was suggesting that drupal_render() could be responsible for making rendered elements "drillable", with the process phases and then theme() is only responsible for invoking the right theme/template function (which couldn't happen until #2006152: [meta] Don't call theme() directly anywhere outside drupal_render()) and so we could do the things like what '#type' => 'link' does and inline rendering inside drupal_alter() for speed.
Even if that doesn't happen though, if you had '#type' => 'link' which renders a string into '#markup' in the '#pre_process' you don't want drupal_render() to call theme_link() so you would set '#theme' => FALSE in hook_element_info() as otherwise '#theme' => 'link' would be assumed.
Well, probably nothing at the moment. Potentially though, if #type determined the preprocessing and not #theme, in contrib/custom you might want to do something like '#type' => 'link', '#theme' => 'my_crazy_link' to send the "base" processed/sanitized variables off to your own crazy link theme function/template.
Possibly not but I feel like that line of enquiry should be a followup issue as trying to do that here could derail this first step.
Comment #4
tstoecklerJust a note that '#type' => 'table' and '#theme' => 'table' are two *very* different things currently. I totally applaud striving for consistency and I have suggested moving over to '#type' => 'table' for all tables before, but this will be a non-trivial effort.
Comment #5
thedavidmeister CreditAttribution: thedavidmeister commented#4 - if they're legitimately two different things, why don't we split them into two types rather than trying to force them to be merged?
Comment #6
tstoecklerNo, they are the same thing - an HTML < table > - but they are currently declared differently. Example:
Comment #7
thedavidmeister CreditAttribution: thedavidmeister commented#6 - sure... I think I'll update the issue summary because that's just another example of why we *should* make the change.
I wonder if it would be possible/easier to give '#type' => 'table' a pre_render callback that "magically" transforms the current type syntax to theme syntax and always passes it off to theme - at least as a stopgap measure. We could file a followup task to actually refactor everywhere tables are called and remove the magic later.
Comment #7.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #8
thedavidmeister CreditAttribution: thedavidmeister commentedWow, I just realised that for '#theme' => 'item_list', '#type' means "type: The type of list to return (e.g. "ul", "ol")."
Comment #8.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #9
Fabianx CreditAttribution: Fabianx commentedYeah, that should really be #list_type instead ... Issue?
Comment #10
tim.plunkett#1828536: Rename 'type' variable of theme_item_list() to 'list_type'
Comment #10.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #11
thedavidmeister CreditAttribution: thedavidmeister commentedBased on c4rl's git repo I'm going to add in the proposal that if #type is not set in our array we simply concatenate the values and return. This allows partial compatibility with existing array('#markup' => 'foo') style arrays and gives us some kind of simple default fallback behaviour.
Comment #12
thedavidmeister CreditAttribution: thedavidmeister commentedCurrently if theme($elements['#theme'], $elements) returns an empty string then drupal_render() tries to render $elements['#children']. I don't understand the reasoning behind that, it looks like a bug to me so I'm going to add to the issue summary:
If theme() is called then theme() is 100% responsible for rendering both the element and it's children, drupal_render() will not interfere with or "fallback" to attempting to render #children once theme() has taken this responsibility.
Comment #12.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #13
thedavidmeister CreditAttribution: thedavidmeister commentedAnd currently if '#theme' is set, the behaviour of drupal_render() interpreting '#markup' changes as well! That's something that goes against this issue too. Updating summary with:
'#markup' is a reserved key for drupal_render() and its value will *always* be included in the output regardless of whether '#theme' is set, theme functions are therefore advised to not define a 'markup' parameter. The final output is always $prefix . $markup . $rendered . $suffix.
Comment #13.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #13.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #14
thedavidmeister CreditAttribution: thedavidmeister commentedHere's a patch that just puts the current proposal into drupal_render() without attempting to address any inevitable fallout or update much documentation. Setting to needs review just so we can see the damage ;)
Comment #15
thedavidmeister CreditAttribution: thedavidmeister commentedFollowup/related - #2009152: The default behaviour for drupal_render() when theme() is not called could be better at representing HTML as structured data
Comment #15.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #15.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #15.2
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #16
tim.plunkett#markup with no #type means that #type is 'markup.
#markup respects #prefix and #suffix and everything else.
Comment #17
thedavidmeister CreditAttribution: thedavidmeister commented#16 - Fair point.
@Fabianx was the one telling me he was looking for a simple and fast way to check for "is renderable array" (ie. without calling an drupal_is_renderable_array() function) so I'd like to hear if he's happy with if (isset($foo['#type']) || isset($foo['#markup']) outside of drupal_render().
Comment #19
thedavidmeister CreditAttribution: thedavidmeister commentedWhat about this?
If #type is not set then markup is assumed and #theme is set to FALSE. This allows the existing simple array('#markup' => 'foo') style renderable arrays but enforces that any renderable array wanting to use theme() has to set a base #type and provide #theme only as an override for the default callback.
Comment #19.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #20
thedavidmeister CreditAttribution: thedavidmeister commentedalso, if anyone is wondering about the new elseif logic, #render_children is set by theme()
Comment #21.0
(not verified) CreditAttribution: commentedUpdated issue summary.
Comment #22
thedavidmeister CreditAttribution: thedavidmeister commentedComment #23
thedavidmeister CreditAttribution: thedavidmeister commentedI added:
in an effort to reduce the immediate impact on the API of this issue and reduce the amount of work required here.
I also started cleaning up hook_element_info() implementations that I could find.
Comment #24.0
(not verified) CreditAttribution: commentedUpdated issue summary.
Comment #24.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #24.2
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #25
thedavidmeister CreditAttribution: thedavidmeister commentedI opened #2010212: convert all #theme => table with #type => table and #2010210: Convert all item_list #theme to #type
Comment #25.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #26
thedavidmeister CreditAttribution: thedavidmeister commentedfix for a fatal in #23
Comment #27.0
(not verified) CreditAttribution: commentedUpdated issue summary.
Comment #28
thedavidmeister CreditAttribution: thedavidmeister commentedmark does the same thing as item_list by using #type for input. Updated summary.
Comment #28.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #28.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #29
thedavidmeister CreditAttribution: thedavidmeister commentedI split a small part of this patch out into its own issue #2012800: \Drupal\Core\Render\Renderer::doRender() could be lazier about calling Element::children()
Comment #30
thedavidmeister CreditAttribution: thedavidmeister commentedanother small part of the patch I'm splitting out #2012812: drupal_render() can't distinguish between empty strings from theme() and no hook being matched
Comment #31
thedavidmeister CreditAttribution: thedavidmeister commentedsplitting out a third bit #2012818: Remove #type 'markup'. Whittling the scope of this patch down :)
Comment #32
moshe weitzman CreditAttribution: moshe weitzman commentedLets disregard Doxygen bugs and various implementations that do things wrong. The relevant ideas of drupal_render() in my mind are:
So, it is natural that drupal_render() does not care about #type because #type is not part of its API. When #item_list uses #type, it does so correctly because #type is not a reserved word.
We can certainly change how things work. I still think it is useful share understanding about our starting point. I'd be happy to do a phone call on drupal_render() topics if thats helpful.
Comment #33
thedavidmeister CreditAttribution: thedavidmeister commented#32 - Yes, this issue is looking to change how things work. To make #type required in renderable arrays and make it determine how drupal_render() behaves (ie. stop doing special casing for renderable arrays based on #markup and #theme beyond picking a theme function) - and therefore make it part of its API.
I'm not sure I totally understand which things in core you'd consider are "doing things wrong" and which things you think are working as intended.
What about when #mark does it? I believe this throws errors - #2009578: Replace theme() with drupal_render() in history module. Part of this issue is a proposal to make #type into a reserved word so that the API doesn't allow us to make theme functions that don't have robust renderable array representations.
For example, I could define a valid theme function that expects to be called like this currently:
theme('foo', array('type' => 'table'));
type here expects 'table' or 'item_list' in this theme function. How would you represent this with a renderable array with the current system without the incorrect defaults being loaded by drupal_render()?
That's not 100% true. #prefix, #suffix, #markup are all handled by drupal_render(), not the theme() function called. You can have a #type that doesn't require #markup to be set when drupal_render() is called, and then it is built during the pre_render "phase" without using theme functions -> see #type => 'link'. You can also set #children in the array that you pass in to drupal_render() and theme() won't be touched unless you have #theme_wrappers set.
That would still be the case, but I'm also proposing that if #theme isn't set then it is assumed to be the same as #type which would mean that for the majority of existing array('#theme' => 'foo') type renderable arrays the conversion would be as simple as changing it to array('#type' => 'foo') and leaving the rest as-is.
Sure. I'm not sure if timezones would be an issue, I'm in Melbourne. An IRC chat is also an option, I'm usually in #drupal-twig when I'm on :)
Comment #33.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #33.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #34
thedavidmeister CreditAttribution: thedavidmeister commentedI added #2015177: Update documentation around hook_element_info() to not say/imply element types are limited to the Form API and removed things from the issue summary that have not been split into their own issues. The patch in #26 now needs a re-roll with reduced scope.
Comment #35
thedavidmeister CreditAttribution: thedavidmeister commentedUpdating summary to more accurately reflect what is being proposed here.
Comment #36
thedavidmeister CreditAttribution: thedavidmeister commentedupdating patch to not cover the issues opened elsewhere.
Comment #36.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #38
thedavidmeister CreditAttribution: thedavidmeister commentedI've been thinking about this a bit more, really trying to reduce it down to something both more achievable for a single patch and with a more palatable scope of API changes.
This issue is still trying to fix two or three different things (down from like 7 but still more than one) which is bad.
1. Wanting to enforce #type for all renderable arrays to make other ideas possible (like a #type based drupal_alter(), for example).
2. Trying to make an explicit workaround around for a bug in drupal_render() where it's relatively easy to call theme() and also the not-theme() fallback, when the intended behaviour is to only ever choose one or the other.
3. Trying to make it both easier to do and easier to learn for developers using the render API to have rich, robust and consistent behaviour across *all* their renderable arrays.
1:
I just don't think we're ready for this kind of change yet and won't be for months, maybe years. #theme isn't even being used everywhere instead of theme() yet, there'd be at least one more complete sweep of core to get everything using #type over #theme, looking at 'table' as an example of this we can see that this can be a "non trivial effort" as @tstoeckler put it.
Additionally, I'm not aware of any of the ideas that require #type being set even having a proof of concept patch available, let alone something ready for review that's complaining about being blocked on this API change.
When/if the time comes to enforce #type, we can just open a new issue for that one thing.
2:
Let's fix that bug elsewhere, there's already tickets open for it #2012812: drupal_render() can't distinguish between empty strings from theme() and no hook being matched, #2015311: If #theme suggestions are set as an array for theme() but none are implemented #markup is treated incorrectly by drupal_render(). It's a little gnarly as it will probably require at least a minor API change to theme() to fix, but the solution will hopefully be better than having to explicitly set #theme = FALSE for over half the #types currently in core.
3:
This is something that could be done here. What about if we just aim to add #type as a theme suggestion for all renderable arrays if #theme is not set? If the theme suggestion isn't available then theme() will just return early and if the bug in 2 is fixed, drupal_render() should know at this point that we're supposed to fallback to the non-theme way of processing this array without needing to be told with #theme = FALSE style declarations.
Anyway, just wanted to write down what I was thinking about somewhere..
Comment #39
thedavidmeister CreditAttribution: thedavidmeister commentedupdating issue title to reflect #38
Comment #40
thedavidmeister CreditAttribution: thedavidmeister commentedjust whittling away...
Comment #41
thedavidmeister CreditAttribution: thedavidmeister commentedI uploaded the wrong patch >.< it's missing a few modifications to existing hook_element_info() implementations.
Comment #43
thedavidmeister CreditAttribution: thedavidmeister commentedThis can't be done until #2012812: drupal_render() can't distinguish between empty strings from theme() and no hook being matched is resolved, postponing.
Comment #44
thedavidmeister CreditAttribution: thedavidmeister commented#40: 2005970-40.patch queued for re-testing.
Comment #46
thedavidmeister CreditAttribution: thedavidmeister commented#40: 2005970-40.patch queued for re-testing.
Comment #47
thedavidmeister CreditAttribution: thedavidmeister commentedhopefully #1985470: Remove theme_link() being closed means less than 147 fails.
Comment #49
thedavidmeister CreditAttribution: thedavidmeister commentedthis just incorporates the latest from #2012812: drupal_render() can't distinguish between empty strings from theme() and no hook being matched.
Comment #51
jenlampton#49: 2005970-49.patch queued for re-testing.
Comment #53
thedavidmeister CreditAttribution: thedavidmeister commentedThis is the only part of #49 that is still relevant.
Comment #55
thedavidmeister CreditAttribution: thedavidmeister commentedhurrrrrrr.
Comment #56
thedavidmeister CreditAttribution: thedavidmeister commentedThis avoids setting #theme from #type if #theme_wrappers is set - so we don't break #type elements that only set #theme_wrappers, like the radio elements in the installer. I don't know why this wasn't already breaking in previous patches.
Comment #58
thedavidmeister CreditAttribution: thedavidmeister commentedThe exceptions are from drupal_get_css() causing the theme to initialise on ajax requests when it calls drupal_render(), when it can't because $theme is empty.
This patch avoids that by adding a check for empty $css arrays in drupal_get_css() and hopefully makes what is happening to #type a little more legible.
Comment #59
thedavidmeister CreditAttribution: thedavidmeister commentedComment #60
thedavidmeister CreditAttribution: thedavidmeister commentedThis could add a lot of calls to theme(), which almost immediately return when the suggestion doesn't match, but I think it should be profiled. We could statically cache hooks that don't match or something if it's bad.
Comment #61
thedavidmeister CreditAttribution: thedavidmeister commentedi just opened #2032967: AssetResolver::getCssAssets() should not try to sort and optimise if $css is empty. so the patches here can stay focussed.
Comment #62
thedavidmeister CreditAttribution: thedavidmeister commentedI had a bit more of a think about this, and I wanted to try changing $theme_is_implemented to start with #theme being !empty() rather than isset(). This way, if profiling showed that a particular #type was generating too much of an overhead by calling theme() a lot of times you could simply set #theme FALSE in hook_element_info() and that would be that.
Comment #63
thedavidmeister CreditAttribution: thedavidmeister commentedComment #63.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #64
star-szrI'm still up for doing some profiling on this, may happen this weekend.
Comment #65
thedavidmeister CreditAttribution: thedavidmeister commentedOk, cool. If you want to do profiling, can you let us know what hooks are the slowest? I'm happy to drop a #theme => FALSE in hook_element_info() for anything that's noticeably slow.
Comment #66
star-szrI won't have time in the near future to profile this after all. The patch does look sensible to me after reading the issue summary though.
Comment #67
Fabianx CreditAttribution: Fabianx commentedThis is RTBC by me, but it needs profiling.
This is especially important for things that do not go via the Form API and an important step for a Theme Component Library.
etc.
Comment #68
thedavidmeister CreditAttribution: thedavidmeister commentedSo, in preparation for some profiling, I put the following code inside the new block of logic that adds #theme:
I used devel to create 50 nodes with up to 4 comments each and displayed them all in the "full" view mode with comments displayed in the front page view.
The result was this:
So, I'm going to re-roll the patch with #theme => FALSE in the element info for "contextual_links_placeholder", "value" "link" and "html_tag" before going through the process of profiling this as I think it's obvious enough that we don't need to be spamming theme() for elements with those #types.
Comment #69
thedavidmeister CreditAttribution: thedavidmeister commentedAfter clicking around the site a bit with the counts on, here's the patch that I think should be profiled.
Comment #70
thedavidmeister CreditAttribution: thedavidmeister commentedneeds review just for the testbots, still needs profiling.
Comment #71
thedavidmeister CreditAttribution: thedavidmeister commentedHere's my profiling results for the patch in #69, 50 nodes displayed with up to 4 comments each on the front page:
Seems fine to me...
Comment #72
thedavidmeister CreditAttribution: thedavidmeister commentedThis might be more useful, I got an API key from Fabianx to get the runs uploaded to lionsad.de:
Comment #73
thedavidmeister CreditAttribution: thedavidmeister commentedNeeds tests though.
Comment #74
Fabianx CreditAttribution: Fabianx commentedUnfortunately this will need re-profiling as this discovered a critical core bug: #2051847: AnnotatedClassDiscovery::getDefinitions is critically slow when viewing nodes with comments
Alternative: Just use 50 nodes (and don't display comments) or 1 node with 50 comments.
Both scenarios had worked fine for me and would be suitable to show comparable performance.
Comment #75
thedavidmeister CreditAttribution: thedavidmeister commentedyeaaaaah, I waited over an hour for those results. I'll re-profile, but I'm looking at tests now.
Comment #76
thedavidmeister CreditAttribution: thedavidmeister commentedAdded some tests.
Comment #77
Fabianx CreditAttribution: Fabianx commentedhttp://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=51f3c666a1782&...
http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=51f3c666a1782&...
Comment #78
Fabianx CreditAttribution: Fabianx commentedMy only question before RTBC is:
Does it make sense to exclude #theme_wrappers here?
Comment #79
thedavidmeister CreditAttribution: thedavidmeister commented#78 - In the current codebase it does because there are types in core that have the name of theme hooks intended as #theme_wrappers rather than #theme.
See comment #56 where I discovered that #type 'radios' sets #theme_wrappers array('radios').
The intention of #2025629: [meta] Ensure that all #theme/#theme_wrappers hooks provided by core are associated with a sensible, re-usable element #type is to clean up anything confusing about the existing #type/#theme/#theme_wrappers namespaces and is well beyond the scope of this issue, so for now we can't support this "auto theme" behaviour on elements that define #theme_wrappers.
As I said on that issue, this is a bit "chicken or the egg" as getting this patch in makes it easier/neater to clean up the existing element/theme definitions while cleaning up the existing element/theme definitions may allow us to support #theme_wrappers here.
There's also this outstanding issue #2042285: #theme_wrappers should be able to have a unique set of variables per wrapper hook that could easily lead to unexpected/undesirable behaviour when adding #theme into an element that already has #theme_wrappers set.
In the end I didn't see the #theme_wrappers support as critical to what this patch was trying to achieve and right now it causes more problems than it solves with various conflicts. I'm sure this is something we could revisit in D9 if/when we get our theme system structured a bit better overall but for now I think we can play by the 80/20 rule :)
Comment #80
star-szrThis is looking good and I can see why #theme_wrappers were excluded. Great work @thedavidmeister and @Fabianx!
Comment #81
Fabianx CreditAttribution: Fabianx commentedMakes perfect sense.
* A good usage for cleanup.
* A nicer integration of #type (FAPI) and #theme (theme system)
* Backwards compatible.
* Perfect integration with other theme system architecture changes.
Has tests, performance profiling looks good, all feedback addressed => RTBC
Comment #82
moshe weitzman CreditAttribution: moshe weitzman commentedAny chance someone could update the Issue Summary? SInce it doesn't follow the standard template, I suspect it isn't current. If I'm wrong, please remove the tag and change status.
Are html_tag and value really commonly called, as the comment suggests. html_tag happens once per page. That quite infrequent, given how many theme() calls happen on a page. value is only used for forms? I'm not sure what value means in a drupal_render context, to be honest.
Comment #83
thedavidmeister CreditAttribution: thedavidmeister commentedSee comment 68.
html_tag is called more than once per page. Why do you think it happens only once per page?
You can do a very simple experiment yourself. Apply the patch in #62 so you don't have any of the modifications I made to element_info() definitions and then change this:
so that it contains
print $elements['#type'];
You should see 'html_tag' appear quite a few times, as well as a few other types.
From memory, I believe that I was seeing 'value' most on pages with forms, but that includes comment forms (wywiwygs, etc..)
The changes I made to the hook_element_info() were made based on measuring which #types would hit theme() the most, using the method outlined in #68 - this experiment should be entirely repeatable :)
Comment #83.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #83.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #84
thedavidmeister CreditAttribution: thedavidmeister commentedI have updated the issue summary to use the standard template as per #82.
Setting back to RTBC as per #82 and #81.
Comment #84.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #85
xjm#76: 2005970-76.patch queued for re-testing.
Comment #87
thedavidmeister CreditAttribution: thedavidmeister commented#76: 2005970-76.patch queued for re-testing.
Comment #89
star-szr#76: 2005970-76.patch queued for re-testing.
Comment #90
thedavidmeister CreditAttribution: thedavidmeister commentedComment #91
alexpottAfaics we're missing a test for the ability to set #theme to false to bypass the theme.
Comment #92
thedavidmeister CreditAttribution: thedavidmeister commentedI'm not sure how to actually test that from outside drupal_render(), I'll have to think about that more :/
Comment #93
thedavidmeister CreditAttribution: thedavidmeister commentedI don't know how to test exactly what @alexpott is asking for tests on. I started with this:
But then, FALSE is not an implemented theme hook, so it's an identical test to:
Which already exists.
The only external difference between theme(FALSE, ... ) and theme(array('suggestionnotimplemented'), ... ) is that the former results in a watchdog log:
watchdog('theme', 'Theme hook %hook not found.', array('%hook' => $hook), WATCHDOG_WARNING);
I could try doing something with watchdog, but then if this log call is ever removed we will silently lose test coverage.
I'd love some advice on how to move forward here if anyone could lend a hand.
Comment #94
Fabianx CreditAttribution: Fabianx commentedWell, I think you will need to do:
* Implement hook_element_info and return FALSE for #theme.
* Setup a pre_render function to render into #markup.
* Check that theme() is not called by providing a template / theme function that has different markup.
Done.
Comment #95
thedavidmeister CreditAttribution: thedavidmeister commented#94 - ooooh, is *that* what I'm testing. Yeah, I can do that. I think I misinterpreted the requirements.
Comment #96
thedavidmeister CreditAttribution: thedavidmeister commentedI added tests for:
- #type is set and is an implemented theme hook, #theme is FALSE -> should render #markup
- #type is set and is an implemented theme hook, #theme is a theme hook -> should use #theme
- #type is set and is an implemented theme hook, #theme_wrappers is set -> should render #markup wrapped in #theme_wrappers
Hope that helps :)
Comment #97
thedavidmeister CreditAttribution: thedavidmeister commentedsoz, interdiff attached.
Comment #98
thedavidmeister CreditAttribution: thedavidmeister commentedslight updated on a comment.
Comment #99
Fabianx CreditAttribution: Fabianx commentedNice work, that should address the concerns from #91.
Back to RTBC
Comment #100
thedavidmeister CreditAttribution: thedavidmeister commentedNote to self:
thedavidmeister
4:40 webchick: so, I will:
4:40 add an explanation of #theme_wrappers not being touched to the issue summary
4:40 remove existing #theme => #type lines from element_info() hooks
4:40 samhasse1l left the room (quit: Ping timeout: 245 seconds).
4:41 samhassell left the room (quit: Ping timeout: 246 seconds).
thedavidmeister
4:41 workaround the AJAX thing rather than try to fix it and put a comment linking to the proposed fix issue
4:41 typhoniu_ left the room (quit: Remote host closed the connection).
webchick
4:42 Yeah, something like // @todo This needs more investigation as to why it causes exceptions. http://...?
Comment #101
webchickSo I am not on board with transitioning everything that's currently a theme function to hook_element_info() and thereby creating hundreds of #types as was in the issue summary prior to this evening, at least without much further discussion. To me, hook_element_info() is for a very special case: it defines re-usable components that other modules can make use of in order to build consistent UIs that behave the same across the Drupal product. If #types button, checkboxes, radios, etc. start getting conflated with #types aggregator_wakka_wakka_foo, that's going to create a huge learnability problem in and of itself. It needs to be really easy for new developers to understand what elements they're recommended to use for the 90% case.
So I've asked thedavidmeister to postpone #2025629: [meta] Ensure that all #theme/#theme_wrappers hooks provided by core are associated with a sensible, re-usable element #type until/unless a more holistic plan is developed there and agreed to by one or more core maintainers. Currently it's catch and I scratching our heads, so this still still needs some work. Remember that we are in the phase of release where each thing we commit need to move us further *towards* release, not further away. New initiatives that spawn 50 more sub-issues of work that risk leaving D8 inconsistent with itself if not 100% complete, and breaking contrib module developers' code unnecessarily is not doing that. Any DX improvements introduced need to be weighed against those types of impacts.
With that out of the way, reviewing this patch on its own merits.
Hm. This looks very, very wrong to me. drupal_get_css() by definition should only be called when there is CSS to put on the page, and never for AJAX requests. I think this needs more discussion on its own in https://drupal.org/node/2032967 so don't really want to commit it here.
The first and last sentences here make sense. I don't quite understand though what "This means that if #type is not implemented as a theme hook then theme() will simply ignore it and return early. " means. Rather than explaining to me how it technically works, can you explain what the impact on me would be as a module developer?
This whole section might go better if it were something like:
"The #theme value defaults to the #type value if it's not already specified, and if #theme_wrappers is not set: This means that an element with '#type' => 'toolbar' will by default try to output its contents with theme_toolbar(). This behaviour can be overridden by specifying #theme or #theme_wrappers, or by explictly setting #theme to FALSE in the case of #types that will be called multiple times on the page."
I dunno, that needs some work. But basically, I'd try and aim the docs more at the consumer of this API and less at another core developer.
It concerns me how often this hunk is in the patch, and how random the elements it's placed on are (contextual_links_placeholder? Really? That would *never* in a million years enter my mind as one of the top 5 theme functions called on a page.).
It seems like it's *very* easy for a module developer to inadvertently introduce a performance regression by not changing their hook_element_info() to match this. This therefore represents an API change; before they'd have to explicitly set #theme to something in order to incur that performance impact. I didn't see the perf #s for what happens if something like theme_value is missing this #theme => FALSE thing. How bad are we talking here?
It's not necessarily that we can't do the API change, but we need to understand the impact it has and make sure the DX win here is enough to tip the scales. And to that DX benefit, one of the main reasons to do this is so we can remove duplicate #type => 'foo', #theme => 'foo' calls everywhere. But this isn't done in this patch. I think that should be added so we can compare before/after code in a more holistic way.
These tests are lovely! Great job.
Comment #102
thedavidmeister CreditAttribution: thedavidmeister commentedok, back to "needs profiling" for the patch without the #theme => FALSE to ballpark gauge "worst case" regression here.
Comment #103
webchickBtw, on IRC thedavidmeister rightly pointed out that this isn't really an impactful API change; people with D7 modules will have one of #theme, #theme_wrappers, #markup, etc. defined already because it was required then.
Who it would impact is module developers from here on out who use core as an example of what to do, and miss that there is an extra step they need if their elements are going to be on the page many times. I still think that "worst case" profiling would be good to do though, so we understand the impact of missing that detail.
Comment #104
thedavidmeister CreditAttribution: thedavidmeister commentedFollowing on from #103, anything defining #theme, #theme_wrappers or #markup will still not be affected by any performance regression.
It isn't, it's one of the top 5 renderable types that requires #pre_render to be rendered. I don't think it's that "random" if you consider that most renderable element types do not require #pre_render to create #markup, so most element types are not candidates for #theme => FALSE here. The performance of all theme functions that actually exist (#theme or #theme_wrappers) is unaffected by this patch.
Only module developers implementing the #pre_render -> #markup pattern (that is a bit of a hack anyway, see #2052253: [META] Add #render property to drupal_render() and convert #type "#pre_render -> #markup" calls to use it) would be affected and only module developers that are already concerned about the performance of calling theme() should even be considering #pre_render -> #markup as a way to implement rendering some markup - you would hope that someone this "deep" already can read the docs and see that setting #theme => FALSE is explicitly recommended for the exact outcome they're trying to achieve.
I will still get some profiling happening, but I want to ensure that the scope of who/what the performance overhead could possibly effect is not misunderstood or exaggerated - I suppose that's exactly what profiling is though ;)
Comment #105
thedavidmeister CreditAttribution: thedavidmeister commentedI have opened #2067759: theme() could statically cache some of it's internals to avoid making function calls in an attempt to get the overhead of calling theme() with an unimplemented theme hook down to a bare minimum.
Comment #106
thedavidmeister CreditAttribution: thedavidmeister commentedI'm glad that @webchick asked for this patch to actually remove existing hook_element_info() implementations that are made redundant by the API addition in this patch.
What this uncovered was that there's a bunch of element types that are defined like this:
So I would like to have a look at a patch where drupal_render() only avoids setting #type in #theme if #type is actually not found in #theme_wrappers, rather than backing out the moment #theme_wrappers is set - which is likely a harmless 'container' or 'form_element'.
What this means is that you would have problems if:
- #type is 'foo'
- #theme is not set
- #theme_wrappers is not set or does not include 'foo'
- theme_foo is actually implemented, but is not intended to be used with #type 'foo'
It is possible, but downright confusing to define a #type 'foo' and #theme 'foo' that are by design incompatible. IMO, if we find an instance of this we should actually clean that up.
Comment #107
thedavidmeister CreditAttribution: thedavidmeister commentedJust as a yardstick on where the DX sits against HEAD, I turned every core module on and there are 55 items returned by Drupal::moduleHandler()->invokeAll('element_info')
There are 8 current clear duplicates, #2025707: Remove unused #theme "file_widget" would make that 9 so, ~ 15% of all types.
A lot of the #types that aren't affected are either FAPI elements using theme suggestions (19 types), like input__textfield for example, or those using just #theme_wrappers and not #theme at all.
Comment #108
thedavidmeister CreditAttribution: thedavidmeister commentedThis patch:
- Removes #theme => FALSE from element_info() definitions, we can profile the overhead of this change
- Will still use #type as a theme suggestion if #theme_wrappers is set, provided #type is not already included in #theme_wrappers
- Improved the documentation by incorporating @webchick's suggestions in #101, with minor modifications to ensure accuracy
- Removes the now-redundant #theme declarations in hook_element_info
- Adds tests for the case that #theme_wrappers contains #type already as either a key or value
- Replaces the usage of #2032967: AssetResolver::getCssAssets() should not try to sort and optimise if $css is empty. with a #theme => FALSE + comment for the type definition of #type 'styles'
Setting to "needs review" for testbot review only.
Comment #110
thedavidmeister CreditAttribution: thedavidmeister commentedI have no idea why this is setting #theme_wrappers to an empty array instead of letting #type => 'details' provide '#theme_wrappers' => array('details')
Looking at the commit history, the empty array used to be for #type 'fieldset', nothing to do with #type 'details', so it looks like it could be a mistake?
Anyway, I'll see what happens if I use empty() instead of isset().
Comment #111
thedavidmeister CreditAttribution: thedavidmeister commentedignore, #110, I did it wrong.
Comment #112
thedavidmeister CreditAttribution: thedavidmeister commentedComment #113
thedavidmeister CreditAttribution: thedavidmeister commentedthis patch should be a little faster, not using array_key_exists(), using isset() instead.
Comment #114
thedavidmeister CreditAttribution: thedavidmeister commentedImproved docs, added a test.
Comment #114.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #114.1
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #115
thedavidmeister CreditAttribution: thedavidmeister commentedAdded the following to the issue summary:
Comment #115.0
thedavidmeister CreditAttribution: thedavidmeister commentedUpdated issue summary.
Comment #116
sunIt appears a lot of thought went into this already. However, after reading through all comments, one fundamental part that I do not get is why you first want to introduce an automated suggestion, and as the right next step, introduce a special magic value to prevent the automated suggestion?
In terms of DX, I wonder how that is an improvement? Instead of:
...it becomes:
Long story short, the "sane magic default" concept only works if the condition for applying it is straightforward.
That is the case in e.g. the drupal_prepare_form() case being referenced in the issue title: #theme is only suggested, if no #theme is set. That's it, there is no further logic.
If there are additional conditions and special behaviors, then a magic suggestion only results in
var_dump()
DX pain — to figure out why something suddenly happened without having asked for it.I'm not sure whether I'm missing something. Could you clarify the proposal regarding these concerns?
Comment #117
thedavidmeister CreditAttribution: thedavidmeister commentedWell, part of the proposal is to discourage having #type and #theme with the same name if they're not compatible with each other.
I'd prefer for this not to be the case, I don't *want* this behaviour and early versions of the patch did not include it. The reason it works this way is because of the blurred, poorly documented line between what's appropriate as a "theme wrapper" and what should be a "theme hook". Things in core like #type 'radios', that adds a 'radios' theme wrapper while setting no #theme make the situation complicated.
I did open #2053671: [meta] Name all theme functions that are only compatible with drupal_render() when nested within #theme_wrappers "foo_wrapper" but I don't even believe that's a good solution either because the naming becomes so awkward :/
Suggestions are welcome >.<
Comment #118
sunComment #119
joelpittetThe intentions behind this issue are great but I feel it may be unintentionally complicating things around those exception cases and needs a few more eyeballs to get behind it, sorry I didn't see this earlier but I think for now we should look at pushing this to 8.1.x as a target to improve this. It shouldn't be to disruptive to 8.x.x branch but definately needs more people to review or make improvements.
Still +1 to making that #type/#theme stuff a bit more sane.
There has been quite a few changes to the theme system that would need some big help re-rolling this for 8.0.x/8.1.x either way. Feel free to move it back if you have some ideas on how to make this simpler to deal with the exception cases.
Comment #120
drintios CreditAttribution: drintios commentedComment #121
drintios CreditAttribution: drintios at Skilld commentedRe-roll of patch #114
Comment #123
andypostPatch needs to use approach from https://www.drupal.org/node/2320115
So no reroll possible
Comment #130
markhalliwellRemoved the snippet about using "array of theme hooks", but the entire IS needs consolidation and updating.