Revaluate the concept of 'extra fields'

As said in IRC, extra fields is currently used to expose all kinds of UI elements on entity forms and displays to the manage fields pages, e.g. contact settings for users. They're not going to become a field so we still need a solution for that.

Removing sprint tag, that is only used for issues that are actively being worked on at the moment and we can't work on this before the NG conversion is done and the field modules updated to work with it.

Agreed with Berdir.

A (possibly large) part the stuff that is currently exposed as an "extra field" could stop doing so if/when widgets and formatters become applicable to entity properties, since they will appear in Field UI as being "fields" (whether EntityFields or configurable fields). But :
- I couldn't say for sure that this will cover everything that is currently an "extra field"
- the premises are still hypothetical at this point :-).

So no, I don't think we're ready to drop the concept right now :-(.

Seems this one have duplicate as feature #1818680: Eliminate hook_field_extra_fields() / Redesign field UI architecture
Probably it's enough to use the issue to implement special kind of Typed data for all extra-fields as was done in #1732730: [meta] Implement the new entity field API for all field types

I would really really miss extra_fields, but I do want them with formatters and settings: #1983398: Allow formatters & formatter settings for extra_fields (display). Currently (D7) that's not possible with clean contrib, so the Field API will have to progress a bit.

With this discussion on hold, is it worth pursuing a fix in a separate issue to the extra fields system as is to actually use the extra field weight values when building entity forms? Right now, the default extra field weights and UI configured weights are completely ignored. A simple example is to move a node type's title extra field to appear beneath the body - it will not move.

Ahh, yeah, Bojan indicated that might be the case. I'd assumed that patch was already in when I started seeing form display variables in the $form_state. That patch is ginormous! I'll apply it while I'm fiddling around to see if it brings up any problems and make sure it fixes the one I know about. : )

So, right, I think 'extra_fields' are here to stay in D8.

Strictly speaking, the handling of extra fields has nothing to do in field.module though.
The corresponding hooks and functions should move to entity.module / the entity system:

Note that storing & loading their display configuration is already taken care of by EntityDisplay / EntityFormDisplay objects.

- field_info_extra_fields() / FieldInfo::getBundleExtraFields() / FieldInfo::prepareExtraFields()
(collect and cache the list of available extra fields)
I would say this would make sense split in two (extra fields in displayed entities, extra fields in forms), and moved to
EntityRenderController / EntityFormController.
- hook_field_extra_fields() / hook_field_extra_fields_alter()
(let modules expose their extra fields)
According to the above, hook_field_extra_fields() would be split in two separate hooks, one for entity views (called by EntityRenderController), the other for entity forms (called by EntityFormController)
Would be named hook_entity_something(), and would be documented in entity.api.php

Would need sign-off from a core committer for the API changes:
- hook_field_extra_fields() / hook_field_extra_fields_alter() renamed
- field_info_extra_fields() renamed (not too likely to be used in contrib though)

Discussed this with Alex Pott, effulgentsia and xjm at the Drupal Developer Summit in Chicago and we felt the extra field stuff should no longer exist. Given that the "extra fields" concept will most likely be removed in Drupal 9, we should just leave it alone for Drupal 8. It's not worth an API change at this stage of the release cycle. I believe we're tracking the proper solution in #1818680: Eliminate hook_field_extra_fields() / Redesign field UI architecture, hence I'm marking this "won't fix".

I do think the concept of "extra fields" is here to stay... "Add an arbitrary render array to an entity_view, and let admins reorder it through the 'entity display UI" is not a need that's going away even with more flexible fields. Not everything can be reduced to "a given formatter applied to a given field".

But I can live with the status quo in D8.

Then, retitling (back) rather than won't fixing.

Not everything can be reduced to "a given formatter applied to a given field".

No, but can't it be reduced to a component once we have #1875974: Abstract 'component type' specific code out of EntityDisplay?

Well, #1875974: Abstract 'component type' specific code out of EntityDisplay introduces component types, not individual components.

The scenario you describe is:
every arbitrary piece of render array currently added in FooRenderController::buildContent() or some flavor of hook_entity_view()/_alter(), becomes its own separate 'component type' plugin class.

- If the actual rendering stays in buildContent() / hook_entity_view() and is not in the plugin classes, then those plugin classes are all the same, and it's weird to have them be separate plugins. The current D8 way of "expose your stuff in a hook somewhere & a single 'component type' plugin will take care of managing their settings" makes more sense IMO.
- If the actual rendering happens in the classes, then what we have is just Blocks :-), and this is the "merge entity display with the block system" approach that was discussed at some point.

So yes, IMO the only thing that would let us remove "extra fields" in D9 is to fully merge "components in a rendered entity (fields, field groups, extra fields, DS 'custom components'...)" with blocks. EntityDisplay then become an entity-level "populated layout", like SCOTCH provides page-level "populated layouts". It was far too big / too late for D8, notably while SCOTCH was still a moving thing up in the air, but maybe for D9...

This means hook_entity_view() & friends become much less useful, and the DX to get "some arbitrary piece of output" into an entity becomes a bit more involved - you need to declare a block.

One challenge that comes to mind (among others...), is that field_groups introduces nesting, and I don't know if SCOTCH planned to account for that.
One other challenge is performance :-). SCOTCH's mission is "display one page layout fast enough". With the above, it becomes "display one page layout + 10s of entity layouts".
[edit - oh, and another challenge was node.tpl becomes basically unworkable]

Suppose the easiest way to move extra fields to calculated and allow entity display use 'em

Extra fields are genius, I'm planning to build a distribution that uses extra fields as a complete replacement for blocks. No more hacking template files or preprocess functions to embed blocks inside nodes! Instead we shall have one simple, unified way of displaying content through view modes.

One year later I still hold the same opinion. Extra fields and field UI can potentially solve the issue of slow block rendering and more elegantly than panels since it's more native to Drupal core. Use field groups and you'll have a completely flexible layout manager without having to create panels layouts in code. What you need to do is create a module that exposes all blocks as extra fields and then create a sweet ipe-like interface to place field groups, normal fields and extra fields/blocks wherever you wish. With the power of render cache and cache tags in D8, it should be really performant. The only thing that I really miss is to be able to create field formatters for extra fields, so that you can output for example author information in different ways. Maybe I'll be able to create a contrib solution for that, we'll see how far I can get within the limits of Drupal core.

@ordermind We do need extra field formatters/settings. Contrib could do this, but it won't be well known enough unless it's provided by Drupal. Relevant: #1983398: Allow formatters & formatter settings for extra_fields (display) and many duplicates

We should see what we can do with this in minor releases.

Adding an extra field from custom code or contrib still requires hook_entity_extra_field_info and hook_entity_view_alter. Also, I couldn't get thirdPartySettings to work on the extra field itself even though it gets initialised with an empty array. If you want to add custom settings you also need a hook_form_FORM_ID_alter and a form submit callback.

Could we wrap all these hooks up into a new plugin type? @ExtraFIeld, that made it easier for developers to add extra fields without having to find all the procedural hooks?

ExtraFieldComponent is good idea

With that approach you have to have one of those dummy field item classes for each field you add? Plus, if you want to format the output differently, you need a new field formatter each time as well. I did actually contemplate this approach myself for a contrib module but ended up using an extra field to place a simple link.

@ivanjaros I think you're genius! Thank you for sharing your solution. It's not perfect but it's far better than current extra field :)
It'd be really great to have something in Core to modernize these concepts.

I think we should something similar to #23 in core as well.

#21 I couldn't get thirdPartySettings to work on the extra field

There's a new contrib module called "Extra field settings" that makes this easier, see this article.

#21 Could we wrap all these hooks up into a new plugin type? @ExtraFIeld, that made it easier for developers to add extra fields without having to find all the procedural hooks?

There's a new contrib called "Extra field" module that does just this.

I totally agree "something" like extra fields, e.g. a field that returns a programmatically derived ("calculated") value, instead of a database value, makes a lot of sense in core and is definitely needed.

Additionally I agree that the current very simple concept of extra fields should be expanded to OOP and using types (e.g. integer, decimal, text, ...) for representation and formatter, while the value is from code and the field itself should be defined in code.

So to sum things up, we need something that at least solves the need for extra / pseudo fields, but there are lots of possibilities to expand and improve the concept.

I totally agree "something" like extra fields, e.g. a field that returns a programmatically derived ("calculated") value, instead of a database value, makes a lot of sense in core and is definitely needed.

Sounds like you are talking about computed fields.

@fago yes, kind of. I think the whole thing I described is around computed / extra / dynamic / pseudo fields. My proposal was a more complete out-of-the-box solution for such dynamically calculated / derived fields in core, without storage.
Guess with computed fields you mean the contrib project, that also allows storing these values then. I think that part should be left to contrib indeed.

TL;DR: I was talking about making such dynamic fields a first-class-citizen in core. hook_entity_extra_field_info() doesn't feel like that in contrast to other Plugin concepts. Maybe it could be summed up with sth. like: Improve Extra Fields in core and use Plugins.

Is that a bit clearer?

@anybody maybe you already know, but computed fields exist in core. Here is an excerpt from entity.api.php:

function hook_entity_base_field_info(\Drupal\Core\Entity\EntityTypeInterface $entity_type) {
  if ($entity_type->id() == 'node') {
    $fields = [];
    $fields['my_module_text'] = BaseFieldDefinition::create('string')
      ->setLabel(t('The text'))
      ->setDescription(t('A text property added by my_module.'))
      ->setComputed(TRUE)
      ->setClass('\Drupal\my_module\EntityComputedText');

    return $fields;
  }
}

Is that what you have thought of or do you mean something else?

I like how "Improve Extra Fields in core and use Plugins" makes it a solvable task.

@mxh sorry my fault, indeed I now remember I saw that long ago, and think I also used it in a contrib project, but forgot about it. Instead, I used hook_entity_extra_field_info() a lot in such cases.

Now I'm indeed unsure, if the Plugin idea still makes sense. Let's all think about that for a while. Thank you for the reminder and sorry I didn't remember that.

Looking at some practical use cases, I don't think computed fields would be an equivalent replacement for extra fields.

Extra fields, as elaborated already in previous comments long time ago, mostly serve UI-related things. That means in many cases you'd want to have something like a renderable array and/or Twig template involved when displaying the extra field's "value".

On the other hand, computed fields are at the level of field definitions, which could be seen as the "data layer" maybe (it's one layer between database storage and frontend layer). I also think computed fields may be exposed automatically such as JSON API since they are "real" fields. So it may come with some surprising or even unwanted side effects when trying to use computed field instead of extra field.

Looking at proper replacements, I currently think extra fields are some sort of relict from the past but at times still needed. They're almost impossible to be reused. But they may be still relevant if you still use classic "Manage display" without Layout Builder (or maybe in future Experience Builder) and "Manage form display". For example, I often have to define them when rendering further markup sections within an content entity form in-between other fields.

When using Layout Builder, I'd prefer creating blocks instead of extra fields. They can already be implemented as full plugins and reused at other places if needed. Blocks are the most close equivalent that come to my mind right now. I had SDC in my mind as well but these are a bit lower level than block plugins (you could still write a block plugin on top of a SDC if you want though). Once Experience Builder is available, it might also come with yet another concept.

One problem is with blocks however is that you may not need that level of reusability. Extra fields can be attached just where needed, and they won't popup at other places as an option. That's why I have one generic "Text" block plugin defined and use it with some powerful input filters. At some places I even have a Twig block plugin defined for being able to directly write an inline Twig template that is meant for one-time usages.

If I'd be able to add an arbitrary "Twig" extra field via "Manage display" and "Manage form display", similar to how Fieldgroup does it, I'd probably never need to write an extra field anymore. But that's just what I currently have on my mind and certainly others have different practical use cases. Maybe we should collect the practical use cases that make extra fields still relevant and then see whether we can derive a modern concept out of these.

edit: my last comment update, I swear.

I've been using UI Patterns, which lets you assign entity values to SDC props/slots. That could be used in multiple ways on a site, so having the value attached to an entity instead of calculated in a block would be helpful.

Adding a related issue.

With manage display vs. layout builder vs. XB/canvas, there is already the issue that the 'Submitted by' information isn't controlled by managed display, and it's also not a single field that can have a formatter. So while it's not an 'extra field' (yet) it's an obvious case in core where we need something that's is a composite of two different fields rendered as one unit.

After some more thoughts I guess the concept of an extra field itself is still valid, even when looking at all other concepts we have now. A plugin-based approach as suggested by @anybody, maybe covering methods for optional configuration and rendering could be straightforward.

Splitting the concept up into "view" and "form" plugins might be worth it, this could make implementations get aligned to single responsibility and core may already prepare as much as it can for rendering where it's supposed to be placed.

Maybe something like this:

#[ExtraViewField(id: 'my_extra_field', entity_type: 'node')]
class MyExtraViewField implements ExtraViewFieldInterface {

  public function buildConfigurationForm(...);

  public function viewElement(array &$element, array &$build): void {
    // $element has #weight already set and is already attached to $build['my_extra_field']
   $element += ['#type' => 'details' ...];
  }

}

#[ExtraFormField(id: 'my_extra_field', entity_type: 'node')]
class MyExtraFormField implements ExtraFormFieldInterface {

  public function buildConfigurationForm(...);

  public function formElement(array &$element, array &$form, FormStateInterface $form_state): void {
    // $element has #weight already set and is already attached to $build['my_extra_field']
   $element += ['#type' => 'details' ...];
  }

}

@mxh nice draft! Maybe we should put that into a separate issue then?

Some additional thoughts on the draft:
1. entity_type(s) parameter:
#[ExtraViewField(id: 'my_extra_field', entity_type: 'node')]
Would it make sense to allow more than one entity type:
#[ExtraViewField(id: 'my_extra_field', entity_types: ['node', 'user'])]

Or even leave out that parameter to have it on all existing entities automatically?
#[ExtraViewField(id: 'my_extra_field')]

2. Optional display parameter:
#[ExtraViewField(id: 'my_extra_field', entity_type: 'node', displays: ['teaser'])]

3. Configure field ui visibility:
Hide the field entirely in the field ui sorting, only make it available programmatically
#[ExtraViewField(id: 'my_extra_field', ..., visible: FALSE]

4. Set "disabled" by default (maybe this should be assumed):
#[ExtraViewField(id: 'my_extra_field', ..., disabled: true]
#3543096: Enable extra fields (hook_entity_extra_field_info) only in "Default" display mode by default
or
#[ExtraViewField(id: 'my_extra_field', ..., region: 'disabled']
So that the field in region "disabled" by default.

Just some ideas to discuss, which I had in the past regarding extra fields... Maybe some of them could alternatively better be methods of the interface?

Another option would be to annotate functions, not classes, like OOP hooks do...?! Or maybe better not...
When using a class, the benefit might be to combine the form display, view display and eventually some kind of data handling in one class?!

Yeah #53 is pretty encouraging and exporting in a new issue sounds good. Moving this one to a plan.

Answering #54:

Yes of course, the attribute constructor arguments may be defined in the ways we see to make most sense, including support of multiple entity types, default visibility settings etc.

Another option would be to annotate functions, not classes, like OOP hooks do...?! Or maybe better not...

OOP hooks make sense to be defined on methods since they address single invokable functions. Extra fields on the other hand may involve more things besides the main rendering / widget logic. One thing that may be often used is building the configuration form. Having all of that resided in one class that belongs to the extra field's functionality would be great. So using a class-level attribute seems suitable.

When using a class, the benefit might be to combine the form display, view display and eventually some kind of data handling in one class?!

This might be possible if an extra field plugin could live and be discoverable within the same (parent?) directory. Then something like this could be done:

#[ExtraViewField(id: 'my_extra_field', entity_type: 'node')]
#[ExtraFormField(id: 'my_extra_field', entity_type: 'node')]
class MyExtraField implements PluginFormInterface, ExtraViewFieldInterface, ExtraFormFieldInterface {

  public function buildConfigurationForm(...);

  public function formElement(array &$element, array &$form, FormStateInterface $form_state): void {
    // $element has #weight already set and is already attached to $build['my_extra_field']
   $element += ['#type' => 'details' ...];
  }

  public function viewElement(array &$element, array &$build): void {
    // $element has #weight already set and is already attached to $build['my_extra_field']
    $element += ['#type' => 'details' ...];
  }

}

However such implementation would use the same buildConfigurationForm(...) for both view and form, not sure whether that might be a bad thing.

Anyways we can continue elaborating all this, maybe in a separate issue as suggested by catch.

Just chiming in to point to https://www.drupal.org/project/extra_field and https://www.drupal.org/project/extra_field_plus which might be useful in this context? happy to open a branch and give access if it's of any kind of use

Thanks @pcambra, same here!

With https://www.drupal.org/project/extra_field_plus you add another important point for using a class:
Allow to add settings for extra fields (e.g. widget / display settings). Guess that should be handled by (inherited) methods.

As @mxh wrote in #50 we should elaborate, where extra fields are placed and how they may interact with widgets and formatters. Maybe that also solves the settings partially.

I agree, extra field plugins might be a (yet missing) intermediate layer between the storage and the formatters / widgets. Then they should possibly be named differently in the future... maybe something like FieldSource, ...?

I think adding settings to extra fields is the wrong way to go.

Using Computed Fields is better - you get something that can use existing FieldAPI formatter, and anything that reads fields (Views, SearchAPI, etc) can work with it.

Thanks @joachim - I agree that might make sense.

Then the question in the context of this issue is, if the computed fields functionality perspectively should go into core (maybe without UI) to allow using it code-wise without having to require a contrib module and replace the (maybe too) less powerful extra fields?
At least we run into many cases in client projects where that would be super helpful and extra fields are in the middle between being helpful, but often not powerful enough. Also I think extra fields are not super self-explaining, so also from the DX perspective a redesign might make sense, which allows formatter and widget use.

I'd vote for a strong Drupal API concept in this area, more powerful than extra fields, long term.

What do you think? Into which direction should this issue go from your perspective?

Computed fields already work fairly well in core -- as it happens I was writing a blog post about how to do this yesterday (https://www.noreiko.com/blog/no-data-no-problem-computed-fields-made-simple).

You need two pieces of code (item list class and field declaration) which is the same as for an extra field (build alter hook and extra field info hook).

What might make sense to go into core is the render array formatter from the Computed Field contrib module, as that then allows you to output a render array from a computed field.

Impressive article @joachim! Thanks!
I'm wondering if that should go into Drupal API documentation somewhere or get listed as "Alternative" on the Computed field module page?

Would be worth much more visibility. Any I agree that could also be a good way to deprecate extra fields.

What might make sense to go into core is the render array formatter from the Computed Field contrib module, as that then allows you to output a render array from a computed field.

Also agree with that!

> get listed as "Alternative" on the Computed field module page?

Good idea - done!