Make #group FAPI / render feature work on all form/render #types out of the box

Important, currently incomplete feature that allows decoupling visual form output from form value structure.

@tstoeckler in #2177469-45: Move node base widgets to the top level of the form:

There are different use-cases here that need to be consider. If I just want to change the #default_value or the #description of a form element this patch makes that easier because I can just blindly to that in $form[$field_name]['#foo'] and don't need to bother with possible nesting.

Another use-case, however, is changing the *output* structure of the form, i.e. moving form elements into different groups, adding form elements into existing groups, perhaps nesting groups into each other, etc. With fieldgroup.module in conrib this is currently a non-trivial task that requires deep knowledge about the different structures and array keys that fieldgroup module uses. It's not as easy as simply setting/altering $form[$field_name]['#group'] or anything similar.

@sun in #1856178-1: '#group' Form API property only works with <details> elements:

We should turn the #group facility into a ultra-generic thing.

Currently, the feature is incomplete and inconsistent. There's no reason why '#group' works with textfields and a single checkbox, but not with selects or multiple checkboxes.

@sun in #2177469-41: Move node base widgets to the top level of the form:

It would be a very good idea to start an effort to convert all forms throughout core to the new concept, so as to ensure and guarantee a good and consistent DX. [...] The form_process_group() and form_pre_render_group() callbacks should be removed and the concept should be refactored directly into FormBuilder.

We may refactor the logic directly into FormBuilder, deprecating the existing callbacks. But as a first step, let's add the callbacks to all elements in ElementInfoManager, then let's see which elements need special consideration (VerticalTabs for sure, but also Container after #2893586: Add #optional support to the Container render element.

Here's a first patch. Let's see how it tests.

Not too bad.

Fixing the test failure.

Now we may also simplify FieldLayoutBuilder::buildForm() a bit, removing the #group processing code added in #2796173-39: Add experimental Field Layout module to allow entity view/form modes to switch between layouts.

This patch landing should also be unblocking #2846393: [PP-1] Investigate alternative approaches to moving fields in FieldLayoutBuilder::buildView().

Furthermore, we may refactor many forms, as far as there are no BC concerns, replacing the value-reparenting with '#parents' by form-reparenting using '#group'. This may be a followup or rolled in. If we opted for rolling it in, we'd immediately get more test coverage for '#group'.

Okay, let's rewind and leave layout builder alone for now.

Patch is working well on a rather complex D 8.7.8 project.
Thanks a lot!

Not working with Ajax callback, if we Replace select options using ajax callback.

Patch from comment 11 helped fix #3199414: #group property not working for select or some other setting types.

Thanks!

Re-rolled # 11 for 9.4.

Just ran into the same issue. The key point is, that quite often a way is required to group fields (and groups) without changing the field / form data structure.

While the nesting functionality works great, when you have full control over the code or even want nesting, everything is all right. But if form values are stored by a third party module or core, you may simply change the display, but not the data structure.

#group seems to be a good approach for that case, where you don't want to change the nesting structure.

Adding the "Needs documentation" tag, once this is available. Currently this only seems to be documented on the Drupal 6 documentation page: https://www.drupal.org/node/262758

I only found this blog article about using #group for this, while the documentation page only shows nesting examples. #tree doesn't seem to be helpful to prevent

Example case: Extend views StylePluginBase and add further fields. Use "details" field group to structure the additional settings, without breaking the form save logic. => Impossible (without manually massaging the nested values).

I wrote a blog post about that with examples: https://julian.pustkuchen.com/en/drupal-9-form-api-grouping-without-affe...

As this isn't really documented, what's the expected behavior and what's the plan for that in core?

The SystemMenuBlock gets around this issue with a #process value on the container elements that calls processMenuLevelParents to remove the container from the form values.

Tested and for the moment it works well.

Without a Checkbox, for example, works well with a details parent but not a Radios with a details parent.

With the #20 patch it works well with the Radios child.

I think getting this is a big step for Drupal, since as @Anybody says many times you can't modify the form structure but you do need to be able to group it.

Needed to make this patch: https://www.drupal.org/project/style_options/issues/3310055#comment-1519...

@Grimreaper and @Pancho any ideas how we can take this issue forward and perhaps get some frontend framework manager feedback on this?

I think before we make the next coding steps here, we should get some feedback from the core team, what they think about the key aspects?

I've see that this doesn't work with managed_file and i don't know why.

It shows in the right position but when you select a file the ajax doesn't work well and disapears.

New patch to support managedFile and Ajax Callback.

I think this is not the best way to do it, but now al least works in my manual tests. I am sure that a person with more control of the core can do it in the most appropriate way.

I've see that the problem was when in the ajax callback to upload the file, the preRenderGroup was not needed and i've make a unset of the #groups in uploadAjaxCallback to skip this function.

New patch, the last one was wrong