Make #group FAPI / render feature work on all form/render #types out of the box

Important, currently incomplete feature that allows decoupling visual form output from form value structure.

@tstoeckler in #2177469-45: Move node base widgets to the top level of the form:

There are different use-cases here that need to be consider. If I just want to change the #default_value or the #description of a form element this patch makes that easier because I can just blindly to that in $form[$field_name]['#foo'] and don't need to bother with possible nesting.

Another use-case, however, is changing the *output* structure of the form, i.e. moving form elements into different groups, adding form elements into existing groups, perhaps nesting groups into each other, etc. With fieldgroup.module in conrib this is currently a non-trivial task that requires deep knowledge about the different structures and array keys that fieldgroup module uses. It's not as easy as simply setting/altering $form[$field_name]['#group'] or anything similar.

@sun in #1856178-1: '#group' Form API property only works with <details> elements:

We should turn the #group facility into a ultra-generic thing.

Currently, the feature is incomplete and inconsistent. There's no reason why '#group' works with textfields and a single checkbox, but not with selects or multiple checkboxes.

@sun in #2177469-41: Move node base widgets to the top level of the form:

It would be a very good idea to start an effort to convert all forms throughout core to the new concept, so as to ensure and guarantee a good and consistent DX. [...] The form_process_group() and form_pre_render_group() callbacks should be removed and the concept should be refactored directly into FormBuilder.

We may refactor the logic directly into FormBuilder, deprecating the existing callbacks. But as a first step, let's add the callbacks to all elements in ElementInfoManager, then let's see which elements need special consideration (VerticalTabs for sure, but also Container after #2893586: Add #optional support to the Container render element.

Here's a first patch. Let's see how it tests.

Not too bad.

Fixing the test failure.

Now we may also simplify FieldLayoutBuilder::buildForm() a bit, removing the #group processing code added in #2796173-39: Add experimental Field Layout module to allow entity view/form modes to switch between layouts.

This patch landing should also be unblocking #2846393: [PP-1] Investigate alternative approaches to moving fields in FieldLayoutBuilder::buildView().

Furthermore, we may refactor many forms, as far as there are no BC concerns, replacing the value-reparenting with '#parents' by form-reparenting using '#group'. This may be a followup or rolled in. If we opted for rolling it in, we'd immediately get more test coverage for '#group'.

Okay, let's rewind and leave layout builder alone for now.

Patch is working well on a rather complex D 8.7.8 project.
Thanks a lot!

Not working with Ajax callback, if we Replace select options using ajax callback.

Patch from comment 11 helped fix #3199414: #group property not working for select or some other setting types.

Thanks!

Re-rolled # 11 for 9.4.

Just ran into the same issue. The key point is, that quite often a way is required to group fields (and groups) without changing the field / form data structure.

While the nesting functionality works great, when you have full control over the code or even want nesting, everything is all right. But if form values are stored by a third party module or core, you may simply change the display, but not the data structure.

#group seems to be a good approach for that case, where you don't want to change the nesting structure.

Adding the "Needs documentation" tag, once this is available. Currently this only seems to be documented on the Drupal 6 documentation page: https://www.drupal.org/node/262758

I only found this blog article about using #group for this, while the documentation page only shows nesting examples. #tree doesn't seem to be helpful to prevent

Example case: Extend views StylePluginBase and add further fields. Use "details" field group to structure the additional settings, without breaking the form save logic. => Impossible (without manually massaging the nested values).

I wrote a blog post about that with examples: https://julian.pustkuchen.com/en/drupal-9-form-api-grouping-without-affe...

As this isn't really documented, what's the expected behavior and what's the plan for that in core?

The SystemMenuBlock gets around this issue with a #process value on the container elements that calls processMenuLevelParents to remove the container from the form values.

Tested and for the moment it works well.

Without a Checkbox, for example, works well with a details parent but not a Radios with a details parent.

With the #20 patch it works well with the Radios child.

I think getting this is a big step for Drupal, since as @Anybody says many times you can't modify the form structure but you do need to be able to group it.

Needed to make this patch: https://www.drupal.org/project/style_options/issues/3310055#comment-1519...

@Grimreaper and @Pancho any ideas how we can take this issue forward and perhaps get some frontend framework manager feedback on this?

I think before we make the next coding steps here, we should get some feedback from the core team, what they think about the key aspects?

I've see that this doesn't work with managed_file and i don't know why.

It shows in the right position but when you select a file the ajax doesn't work well and disapears.

New patch to support managedFile and Ajax Callback.

I think this is not the best way to do it, but now al least works in my manual tests. I am sure that a person with more control of the core can do it in the most appropriate way.

I've see that the problem was when in the ajax callback to upload the file, the preRenderGroup was not needed and i've make a unset of the #groups in uploadAjaxCallback to skip this function.

New patch, the last one was wrong

I also encountered this extending a field widget formatter. The patch in #34 seems to work with Drupal 10.4.2

I've needed to group the actions container and came to the following workaround

      $form['actions']['#group'] = 'off_canvas_sticky_header';
      // remove when https://www.drupal.org/project/drupal/issues/2190333 fixed.
      $form['actions']['#process'][] = [Actions::class, "processGroup"];
      $form['actions']['#pre_render'][] = [Actions::class, 'preRenderGroup'];

The current patch needs to be converted to a Merge Request.

That MR looks buggy. The patch in #34 seems to also work with Drupal 10.5

a new patch for Drupal 11.2.2 and PHP 8.4

Update patch #42

I see that the issue has the tag Needs documentation.

¿What means this? ¿There are some documentation to know what are the steps to make this documentation?

I have rebased the MR onto main and fixed the callbacks, the textarea attachments and re-added the container callback that went missing for some reason.
Also added some more tests and fixed /core/tests/Drupal/KernelTests/Core/Render/Element/WeightTest.php that failed because the MR adds the group callbacks now for all elements which changes the test expectations.

Attaching a static patch file against Drupal 11.4.7.

Generated with the help of an LLM.

@berliner thank you very much, that looks good!
Do you think other tests are needed? At least the issue summary says so, looks like it needs an update?

I have added additional test coverage for the AJAX select case reported as failing in #2190333-16: Make #group FAPI / render feature work on all form/render #types out of the box. Test-only change that switches select options multiple times and confirms that the select stays in its group, is not duplicated and correctly submits at the end. This passes locally, let's see what the test bot thinks.

While looking into this I also found something that appears to be an existing core bug that should probably be handled separately: Some elements can be rendered twice when assigned to a group of an element that renders the element again to render its children. I have reproduced this locally on clean main, without this MR applied, with the datetime element. If a textfield is assigned to a datetime element using #group, that textfield is rendered twice. I have added that observation with more details to #2926030: Fully test #render_children rendering and fix/find any bugs.

@anybody I have updated the issue summary with the current state as I see it. I also pushed another commit, improving the managed file ajax update cycle as the original fix for that issue in #33 kind of worked, but had the side effect that other elements grouped inside a managed file widget disappeared after an ajax update.

I don't see anything else that requires tests for now, but let's see what other reviewers find.