In drupal_render(), element_info is not merged recursively, preventing #type from serving its purpose

Why does #type 'form' work though?

Clarified issue summary. Better title. This cannot be backported.

We have a helper function for the "outside scenario": http://api.drupal.org/api/function/element_info_property/7 (see callers)

I guess I'd be fine with unconditionally adding 'main' as an additional layer (+ tests) as a stop-gap solution. Not backportable. Essentially, however, this merely defers/delays the debate and solution we need to have about merging "default" element properties. But anyway, guess I'd +1 the stop-gap fix by now.

The answer of "hey hipster, you should have known that #type defines #foo and #bar - not in the element info, you know, but somewhere else in the page rendering process - so you need to also apply *those* defaults in your page callback!" is an unacceptable answer.

I'd rather like to see this fixed now, instead of being forgot and still not resolved for D8. Since even the stop-gap fix is not backportable.

So let's make sure that we fix what we can fix easily in order to prevent annoyance in the long run. *Then* attempt to fix it properly. Should be a matter of minutes - especially since the current patch passed already.

In essence: is the block containing output the same element as the block?
or is the output a sub-element of the block?

Right now core tries to do the former, mixing the output into the block.

well 'main page content' being a required block is what really messes with this

Still a block. Of course, if you disable block.module, we run into a larger debate ;) Then, we speak about "containers holding the output of atomic pieces on a page".

Question remains the same though - is it valid/legit to merge the result of the handler responsible for the container /into/ the definition of the page element that makes up the container?

No, hence trying to move it into a sub element of system_main.

Forget about #type container, this affects everything.

well right, but when does it mainfest? why DOES type form work?

Yeah, since #type 'form' also uses #theme_wrappers, that's a detail I'd still like to see clarified in the issue.

So, #type 'form' works, because drupal_prepare_form() does

  $form += element_info('form');

upfront!

Likewise, that page callback returning a container would work equally, if the full element info would be pre-populated upfront.

Meanwhile, I'd agree on adding the 'main' layer for D8 there.

But given this analysis, for D7, I wonder whether we can't squeeze a

if (isset($page_callback_result['#type']) {
  $info = element_info($page_callback_result['#type']);
  if ($info) {
    $page_callback_result += $info;
  }
}

into a function somewhere?

That "fix" seems to have been completely unintentional.

1) I'd still like to see a unit test, which ensures that this works; regardless of version.

2) In light of #9, I wouldn't lose hope just yet. A fix along those lines would look "backportable" to me (along with the test).

Starting point.

Hm. I wonder whether #1473548: CSS classes in $page['#attributes']['class'] are not applied to HTML BODY is related to this or not. (definitely different element/scenario, but seemingly similar situation in that the problem affects the "top-level" render element only)

Looks like this one no need reroll, because it's applied to branch 8.7.x

I think this should either mark this as "works as designed" or bumped to 9.x.

Recursively merging element info now would, technically, be a massive breaking change and would likely introduce duplication callback hell for contrib/custom code.

To give you an example, it's quite common to see the following due to the current limitation:

  $elementInfo = \Drupal::service('element_info');
  $build['container'] = [
    '#type' => 'container',
    '#pre_render' => array_merge($elementInfo->getInfoProperty('container', '#pre_render', []), [
      'my_custom_pre_render_callback_function',
    ]),
    '#process' => array_merge($elementInfo->getInfoProperty('container', '#process', []), [
      'my_custom_process_callback_function',
    ]),
  ];

This is really just another by-product of using render arrays; how they were originally created.

I've run into this on occasion, but easily fixed this "flaw" by simply placing everything in a wrapper element instead of the top level render array.

$build['wrapper'] = ['#type' => 'container'];

---

Possible dup? #2070131: Form & Render Element default properties not properly merged when property is an array of callbacks

I'm going to agree with Mark here. And dawehner in #2894952: Render element preRenders don't get added if a module form_alters the $element.

This is tricky. The current behaviour allows to remove pre_render functions completely, merging it in would no longer allow that.

It's a deep flaw in Form API. There are tons of places in Form API where you try to make a specific change and it causes a cascade of problems later in the rendering process.

For example, if you use form_alter to make a type-specific change to an element, a later alteration can change the #type and now your change makes no sense.

From my quick count, there are at least 4 places where you can modify a form and late or early changes can muck up the rendering. Let's see…

1. The merging of '#type' => 'form' defaults from \Drupal::service('element_info')->getInfo(). These defaults can be altered by a module or theme with hook_element_info_alter.
2. A module's hook_form_alter or hook_form_BASE_FORM_ID_alter or hook_form_FORM_ID_alter.
3. And then a theme's hook_form_alter or hook_form_BASE_FORM_ID_alter or hook_form_FORM_ID_alter. I mention this separately because a module's implementation always comes before a theme's implementation.
4. The #process functions
5. The #after_build functions
6. The #pre_render functions on the form. The list of these functions can be altered by any previous form alteration
7. The preprocess functions on a form-id-specific theme hook. While the "form" hook is always a #theme_wrappers where the form is pre-rendered, a form can contain a #theme hook based on the form ID or type of form (e.g. confirm_form)and, after a specific template suggestion is picked, the preprocess functions can alter variables and since the form is an un-rendered variable, the form can be once again altered.
8. For each element in the form, the merging of #type defaults from \Drupal::service('element_info')->getInfo(). These defaults can be altered by a module or theme.
9. The #pre_render functions on the elements
10. The preprocess functions for the elements' theme hooks. You can alter the list of preprocess functions in hook_theme_alter

Oh, wait. That's 10 places.

So, merging the element info defaults is only a tiny part of a larger problem. And fixing this tiny problem is way too late; many developers know about this problem and new devs will instantly see the breakage when trying a simple form_alter.

IMO, the only possible fix to this is to completely replace Render/Form API with a next-generation API.

In the meantime, might I suggest documenting all the ways a form can be altered? #2909574: Document how to alter forms when hook_form_alter won't work