Provide a 'Hidden' field widget

#3: your patch was not resubmitted for retesting

Right, sorry.
I was confused by the fact that resubmitted patch passed testst

Makes sense to me.

I'd consider this RTBC, if you convert the hard-coded/custom 'hidden' definition into an actual 'field_hidden' widget plugin that is supplied by Field module itself.

UI impact?

The only UI impact of this change should be an extra item in the widgets select list from field UI.

How is hidden a type though? Its a visibility setting? I don't think loads of people will understand adding a hidden widget?

The Manage Display screen uses <Hidden>, I guess we can do the same here :)

This all seems just a side effect, because the "form" is not a view mode :)

Thanks!

Ugh. I thought Field API would support an "any" field type or similar wildcard flag that disables the field type constraint... :-/

I wonder whether this can be fixed easily or whether we should go back to the first/original patch.

Pinging @yched for that.

P.S.: "- Hidden -" is definitely the correct and proper label for special options (which we should also use in the first patch, in case we go back to that). The <bracket> style option labels are stone-age and should be replaced.

Several things here:

I'm open to finding a way to state that a formatter / widget is applicable to any field type. That would need to be some convention like field_type = array('_any'); in the widget info. (empty array is already taken, it means no applicable field type - that's usually completed in hook_field_widget_info_alter())
Right now, rather than hardcoding all core field types in that list, I'd suggest to keep the array empty in the annotations, and use a hook_field_widget_info_alter() to add all existing field types.

More annoyingly, hiding fields in forms is more complex than hiding fields in displays.

- Hiding a field from the edit form is problematic for required fields. Lets you submit an incorrect entity.
You shouldn't be allowed to use "hidden" widget for required fields - but enforcing that is going to be awkward.

- Just outputting an empty form element is problematic, and there was a back-and-forth dance in D7 around that for fields with field_access('edit') == FALSE:
#636834: Field revision data messed up when user has no 'edit' access on the field
#629252: field_attach_form() should make available all field translations on submit
#822418: Field form structure incomplete if field_access() returns FALSE

In short, if the incoming form values contain no values for the field, this breaks new revision creation IIRC, because the values of the "hidden" field are not reported in the new revision.
Thus, for fields with no field_access('edit'), what we currently do is put the whole widget, with #access = FALSE.

But then, there is the case of "existing field value is 2, then the admin changes the field settings so that min allowed value is 5, then suddenly users without edit perm on this field get a validation error they can't do anything about, because the submitted form values are now incorrect, even though they did not change the field value, and cannot even actually correct it. Stuck".
This was fixed by not reporting field errors on elements where #access = FALSE.

Thus, I'd say the widget should do:

'#value' => $the_current_field_value,
'#access' => FALSE,

instead of just being empty.

Also, we should make sure that the 'empty' widget is always last in Field UI's dropdowns.
Support for a 'weight' entry in widget info was added just for this use case in #1586356: Missing 'weight' support from hook_field_widget_info() makes it impossible to sanely order widgets (that was for the "hidden" widget provided by [edit] @dww's @Damien's http://drupal.org/project/field_extrawidgets)

Which, BTW, hints that, since as pointed in the OP, there are many contrib modules trying to fill the need for "hidden widget", it might be a good thing to check how they deal with the issues I pointed in #17.

Hidden either means

- Using access false so people can't tamper with the data
- Or really having a hidden element because maybe some javascript (or ajax or a submit if javascript/ajax isn't there) is going to add some values form another widget selection

Which one do we choose ? Or do we let people configure this ? Depending on the use case, both are possible.

More over, the usage of the contrib modules is low and there's no effective use case for us in core to use this widget at all. And I also think this is going to confuse the hell out of people - sounds ironic coming from a developer I guess :) I'm personally in favor of keeping this in contrib and otherwise hook_form_alter() is our friend.

Bumping to task. For user pictures to properly use fields and not be special-cased (i.e. no more variable_get('user_pictures'), we need a way to disable the form element without losing the data, and amateescu pointed out this issue in irc.

So how about adding a 'hidden' region like on the manage display screen ? I've been doing this for years now with Display Suite and potentially opens up 'view modes' on forms. *running away hard now*

Note that typically doing #access = FALSE on file/image fields means this: #1205822: File(s) silently deleted when #access=false.

Hah. The amount of related issues and monster pitfalls pretty much clarify for me that this should absolutely be figured out by core developers. ;)

@swentel:
I guess a 'hidden' region would basically translate into something very similar to the original patch in #1; i.e., special-casing the 'hidden' widget? I assumed an actual HiddenWidget plugin implementation to be architecturally cleaner, but now with all of those mentioned pitfalls, I could also imagine that a built-in, special handling for widget-less/non-exposed fields might be easier or even required...

It is definitely related to view modes for entity forms, but I'd recommend to not make that a dependency, since the two things don't really seem to be bound to each other.

The thing is that "hidden" on display just means "nothing to do" - so, no formatter = no code fires, fine.

"hidden" on form means "there is some minimal legwork to do, which basically is WidgetBase::form()" (this method also takes care of prefilling default values on entity creation form, which is something we do for fields with access = FALSE, and that we'd thus want to do on "hidden" fields as well). So, "hidden" being an actual widget makes some sense.

So, while I do of regret having "hidden on forms" and "hidden on display" treated differently, it kind of makes sense code wise :-/.
I'm open to discussion though...

Hm. I don't understand how a HiddenWidget could insert default values? — If you configure no widget (the HiddenWidget), then I don't think you are able to enter a default value in the first place, since, yeah, there's no widget...? :)

All you could reasonably do is to iterate over the defined schema #columns and turn those into corresponding #type 'value' sub-elements... with NULL values by default, and only for pre-existing entities/fields, the existing values would be taken over as-is...?

I don't understand how a HiddenWidget could insert default values?

That's how "default values" currently work for fields with field_access == FALSE - or for a programmatic entity_save with no incoming $entity->field_foo entry.
On *creation*, the default value gets assigned; on *edit of existing*, the current value (possibly empty) stays untouched.
So this would need to be the behavior for "hidden" fields as well.

Although, contrary to what I wrote in #24, that behavior might actually be enforced in field_default_insert() anyway, so the form side of a "hidden" widget might not need to care about it.
So that's one argument less for treating "hidden on forms" as a widget rather than as a special 'hidden' case baked in Field API's form handling (i.e more akin to "hidden on display")

Here is a straight port of the existing code from Field extra widgets (which, btw, is from @Damien, not @dww). Tested manually and it works for all existing use cases that were mentioned in this issue (required fields, changing the number of minimum allowed values for a required field, submitting an entity edit form with a field that already has some values in the db).

I guess what we need now is some test coverage and maybe to bring over the UI for the hidden region from Display settings. I couldn't do the them in this initial patch due to lack of time..

Well, to set a default value you need to have a (editable) widget, so I don't see how that can work with this generic hidden one.

Apologies for the wrong attribution of Field extra widgets - I fixed my earlier comment :-).

Tested manually and it works for all existing use cases that were mentioned in this issue (required fields...)

Hm - the code is dead simple... what does "it works on required fields" mean ?

changing the number of minimum allowed values for a required field

I'm not sure I get what that means either :-)

- Has the code been tested to work fine when creating a new revision ? (i.e the existing value of the hidden field must be carried over to the new revision)
- Has the code been tested wrt default values ? i.e when a default value is configured for the field and it uses the "hidden" widget, creating a new node should result in the default value being saved in the node
- Regarding #28 ("can't set a default value in the fields configuration as the default value field is hidden too"), I guess that would need a special hack in Field UI to switch to the field type's 'default widget' when the current selected widget is "hidden".

Not fully decided which approach I'd favor - a) "hidden" as a widget, or b) "hidden" as a 'region', handled by special-case non-widget code in field form code.
b) is more consistent with how we deal with this on the display side, which might prove easier when we start leveraging the EntityDisplay objects (introduced by #1852966: Rework entity display settings around EntityDisplay config entity) on the form side (and we'll need to if we want configurability for #1510532: [META] Implement the new create content page design)
But b) also means extra-fields can be dragged to this "hidden" region as well, and I'm not sure the existing ones are ready for that case - extra fields exposed on forms currently assume they'll be present in the final form.

I guess I'm fine with getting it in as a widget for now, as possibly reconsider later on.
We'd need tests though - for the cases mentioned above, notably.

Using the default widget in the instance settings form sounds good to me, here's an updated patch for that. Next up, tests :)

what does "it works on required fields" mean ?

It means that there's no validation error on an entity form which has a required field instance that's using the hidden widget.

I'm not sure I get what that means either :-)

I was trying to say that I tested this case:

But then, there is the case of "existing field value is 2, then the admin changes the field settings so that min allowed value is 5, then suddenly users without edit perm on this field get a validation error they can't do anything about, because the submitted form values are now incorrect, even though they did not change the field value, and cannot even actually correct it. Stuck".

[Edit]

- Has the code been tested to work fine when creating a new revision ? (i.e the existing value of the hidden field must be carried over to the new revision)

Not yet.

- Has the code been tested wrt default values ? i.e when a default value is configured for the field and it uses the "hidden" widget, creating a new node should result in the default value being saved in the node

Yep, manually, and it works :)
[/Edit]

P.S. I don't like the name of the optional argument introduced for getWidget() at all.. but couldn't find anything else at this time of night :/

there's no validation error on an entity form which has a required field instance that's using the hidden widget.

Right, and that is a problem :-) From my #17:

Hiding a field from the edit form is problematic for required fields. Lets you submit an incorrect entity.
You shouldn't be allowed to use "hidden" widget for required fields - but enforcing that is going to be awkward.

Required fields are the main conceptual issue when adding the ability for users to hide any arbitrary field on forms, and the main reason CCK and Field API didn't allow it so far :-/

re: getWidget($instance_settings_form = FALSE) :
Yeah, not ideal, but we have no other solution than a hacky one here. Will be cleaner when entity form configuration gets refactored based on EntityDisplays too. 'widget' properties won't be stored in $instance anymore, and you'll be able to generate a form input for an instance using any arbitrary (eligible) widget.
I'd suggest an $allow_hidden param defaulting to TRUE, though.

Thanks for explaining again the issue with required fields, I finally got it now :)

So, here's an updated patch that:
- make the required checkbox disabled and adds a helpful description if the instance is using the hidden widget
- doesn't display the hidden widget as an option in the Widget Type screen if the instance is required
- changes the awful $instance_settings_form parameter to $allow_hidden
- adds tests for the cases specified in #30

I wonder if we should cater for fields defined in code as well by throwing an exception (FieldIncompatibleDefinitionException perhaps) in field_update_instance() if the definition has required => TRUE and uses the hidden widget.

Adding the usability tag for the new description text that is introduced in the instance settings form.

I might have missed some of the discussion, but why can required fields not be required? It probably doesn't make much of a difference, but I would think if the default value isn't empty everything should be fine, no? So I would have expected making a hidden field required simply makes the default value form required.

Alright, I don't like having disabled forms - checkboxes, or anything. They are a bad usability practice, as people often do not understand why its disabled and return to just clicking it often enough to get it to work.

If you can't set this required, lets just not offer them the option. I am confident the 80% usecase, would be to not even look for this checkbox anyway. The other 20% with some logic can probably determine you can't set a hidden field required.

I still think the actual correct way to solve this would be to introduce "Form" as a view mode.

As I said in IRC, in the current Field UI which separates the widget selection from the instance settings form, I highly doubt people will 'quickly' realize why the required checkbox is gone.

I might have missed some of the discussion, but why can required fields not be required?

Err.. wat? :) Why can't *hidden* fields not be required? The answer is in #33.

I think we should allow required fields to be hidden. That just means that they will always fail validation of their entity if they are not filled by some other means (prepopulation, etc.). That's a complete corner case, and it is totally fine. No need to add more complexity where it is not required.

Let's see if Yves has some feedback.

@Damien:

I think we should allow required fields to be hidden. That just means that they will always fail validation of their entity if they are not filled by some other means (prepopulation, etc.). That's a complete corner case, and it is totally fine. No need to add more complexity where it is not required.

Well, AFAIK, the general UX philosophy in core is "don't let users enter inconsistent configuration", or at least two config items that blatantly contradict each other.
What's the problem with this ? :
- if widget is "hidden", then raise a validation error of submit of the "instance edit" form if "required & empty default value"
- if field instance is "required & empty default value", then "hidden" is not available in the widget dropdown on "change widget type" form ?

The default value can come from many other places. The most common use case for an hidden field is pre-populating via another mean (URL parameter, etc.). I'm happy with a warning, if we want to make things more complicated for such a small UX gain. But really, there are many other ways to shoot yourself in the foot with the Field UI, I don't see the need to focus on this particular corner case.

"The most common use case for an hidden field is pre-populating via another mean".
Then this other means involves some code somewhere, which is independent and unrelated to the fact that the field is configured to use a "hidden" widget or not.
In which case, I'd think it would be the role of this code to do its job and hide the widget through hook_field_attach_form_alter() when its conditions are met (like, the URL querystring contains a default value) - no need for a "hidden widget" for that ?

#40 is exactly what I tried to propose, but couldn't explain it as well as @yched did. I totally agree with the assessment.

Any chance to move forward here? It's not entirely clear to me what's left to be done for the latest patch in #34?

Same for me...

So, summarizing:

The discussion stumbled on "do we want to prevent site admins from configuring a field to be all of: a) required, b) without a default value, and c) using the 'hidden' widget".

- in current HEAD, 'required' is currently only enforced through FAPI #required property, which wouldn't trigger with the hidden widget, so this setup would silently save "invalid" entities (a field marked as required is actually empty)
- If #1696648: [META] Untie content entity validation from form validation succeeds, OTOH, 'required' is going to be checked at the property level, and this setup would produce forms that can never validate.

So the question is : do we want to prevent that setup with an error on Field UI form validation ?

@Damien advocates that there are cases where this setup might actually be what you want, because the default value is populated by external code (e.g from a URL param).
- The old nodereference_url used to do that with a specific widget, so not our worry.
- But entityreference_prepopulate works on top of any widget, allowing advanced behavior like: hide the widget completely, disable it, or leave it editable, just filling a proposed value. So, right, while very non-80%, it makes sense for the "hidden widget" to work with that too.

So I'd probably advocate outputting a warning instead of an error : "Field foo is required and uses the "hidden" widget. You might want to configure a default value", but still submit the config.

Thanks Yves for a great summary! I agree with the proposed solution and I'll get back to the patch this week.

Here we go. This patch solves everyone's concerns so I think we're ready here.

We can't really set to RTBC if there are notices right ... from #49

- created a boolean-field with "- hidden -" widget... on the field edit page I got:
Notice: Undefined index: id in Drupal\field\FieldInstance->getWidget() (line 59 of core/modules/field/lib/Drupal/field/FieldInstance.php).
Set to required and chose N/A as default value, saved the field and got the same error 3 times again, no warning.

Interesting that notice is :) It comes from the fact that the field type's default widget isn't converted to a plugin yet. I included a small fix for it in this patch.

The second one, getting the warning when you had a default value specified was just a typo from me :/

I think 'Needs usability review' is a better tag to bring in the UX team :P

Just wondering, in views we use "Excluded from display", is it also "Hidden from display" here?

@Damien advocates that there are cases where this setup might actually be what you want, because the default value is populated by external code (e.g from a URL param).
...

So I'd probably advocate outputting a warning instead of an error : "Field foo is required and uses the "hidden" widget. You might want to configure a default value", but still submit the config.

Coming a little late to the party (and in fact we could/should discuss this in a follow-up), but I think any module that adds that sort of "magic" default value should be capable of altering the UI, and removing such a warning, no?

re @tstoeckler : that was my point in #42 too, but then looking at what entity_reference prepopulate does, I didn't really find a way for it to prevent the form error from being set - or unset it once it's been set.

+++ b/core/modules/field/lib/Drupal/field/FieldInstance.phpundefined
@@ -39,12 +39,31 @@ public function __construct(array $definition) {
+    // Skip the static cache if we are getting the widget for the field instance
+    // settings form.
+    if (empty($this->widget) || !$allow_hidden) {

Let's explicitly mention "the widget used for 'default value' input".

Also, I'm really not fond of that code/logic in here. I might be wrong, but I think modifying the $instance['widget']['type'] in field_ui just before calling its getWidget() should work, and be cleaner - not using the "hidden" widget for 'default value' input is field_ui's problem.

+++ b/core/modules/field/lib/Drupal/field/Plugin/Type/LegacyDiscoveryDecorator.phpundefined
@@ -67,6 +67,10 @@ public function getDefinitions() {
+        if (!isset($definition['id'])) {
+          $definition['id'] = $plugin_id;
+        }

Uh - missing indeed...
I even think we should drop the if (!isset())

+++ b/core/modules/field/lib/Drupal/field/Tests/FormTest.phpundefined
@@ -577,4 +577,62 @@ function testNestedFieldForm() {
+    // Create an entity.

Nitpick : plz add that we're testing the default value.

+++ b/core/modules/field_ui/field_ui.admin.incundefined
@@ -691,6 +691,10 @@ function field_ui_widget_type_form_submit($form, &$form_state) {
+    if ($instance['required'] && empty($instance['default_value']) && $instance['widget']['type'] == 'field_hidden') {
+      drupal_set_message(t('Field %label is required and uses the "hidden" widget. You might want to configure a default value.', array('%label' => $instance['label'])), 'warning');

we should also check for empty($instance['default_value_function']).
(same for the other form)

Other than that, cool, and tests++ !

You're right about field_ui's responsability, and it works very well with just a new $skip_cache argument for getWidget().

Fixed the other issues raised above as well.

@yched asked me in IRC to find out why that $skip_cache argument was still needed, and it turns out that $instance['widget']['type'] = $default_widget['id']; didn't call FieldInstance::offsetSet() (more details here).

Yay ! RTBC if green.

Thanks for the perseverance @amateescu :-)

My conscience wouldn't let me slip that unrelated change from FieldInstance.php :/

#64: hidden_widget-1465774-64.patch queued for re-testing.

+++ b/core/modules/field_ui/field_ui.admin.inc
@@ -691,6 +691,10 @@ function field_ui_widget_type_form_submit($form, &$form_state) {
+    if ($instance['required'] && empty($instance['default_value']) && empty($instance['default_value_function']) && $instance['widget']['type'] == 'field_hidden') {
+      drupal_set_message(t('Field %label is required and uses the "hidden" widget. You might want to configure a default value.', array('%label' => $instance['label'])), 'warning');
+    }

Re #58: Again, I don't want to hold this patch up, but adressing this here anyway.

It would require some changes, but I don't think it would be particularly hard to make this properly alterable from contrib. I would suggest the following: Introduce a validation handler that sets $form_state['#empty_default_value'] based upon the if check above. Then in a second validation handler we check if ($form_state['#empty_default_value'] && $form_state['values']['instance']['widget']['type'] == 'field_hidden') and then set a form error. Then contrib can simply add a validation handler in between that sets $form_state['#empty_default_value'] to FALSE.

Discussed a bit with @yched and @swentel and we agreed that using 'hidden' makes more sense for the plugin id, even though 'field_hidden' is consistent with "prefix by the name of the provider module" rule to avoid clashes, this is very much a special case plugin and it also matches the 'hidden' select key used in the 'Manage display' form.

Re #66:
Maybe we should open a followup to discuss if we want to keep the current warning or turn it into an error? Your approach would work, of course, but it seems a bit convoluted for core to do it and for contrib or custom code to go through all that trouble just to be allowed to save a form..

Agreed with both points in #67, and confirming RTBC if still green.

As far as I can tell, this is a new feature and therefore should likely be bound by thresholds. We're not anywhere close to being able to commit new features atm, but hopefully we will be someday. :)

Nope, this is a task, see #20 :)

I'm not sure this is needed for user pictures any more, at least once [#http://drupal.org/node/1853378] is in - although that still has special-cased theme toggles in the current patch so if widgets could be applied to entity properties perhaps it'd allow for removing those?

Committed 809138a and pushed to 8.x. Thanks!

Fixed documentation standards on commit...

 class HiddenWidget extends WidgetBase {

   /**
-   * Implements Drupal\field\Plugin\Type\Widget\WidgetInterface::formElement().
+   * {@inheritdoc}
    */
   public function formElement(array $items, $delta, array $element, $langcode, array &$form, array &$form_state) {

Created http://drupal.org/node/1968224

Looks good to me.

I just came here after reading read the change notice, which doesn't really tell me how it works, what it does to the html output (an input with type=hidden or nothing at all?) and how I can set the value of a hidden field.

Can someone expand the change notice, or is there a better place to document the behavior?

(Oops, didn't mean to add the tag again.)

Marking "needs work" to get some more eyeballs on this. Thanks for reviewing the change notice, Marc!

Considering there is still change notice work to be done / agreed upon... setting tags / title / priority appropriately

#75 is addressed in http://drupal.org/node/1968224/revisions/view/2644910/2647428. So back to fixed. Thanks @patrickd for the changes.

Thanks Patrick. That's much more useful.

Attached is a the above backported to D7, sans tests. I found that autocomplete fields would not work unless I switched to that widget and then back to the hidden widget.

Ugh, wrong patch, this one is a bit better.

You might need to reupload.

Let's try that again

And again.

And again.

Note: #2060705: Remove the 'hidden widget' plugin, it is actually never used :-)

(doesn't exclude a D7 backport, though)

The exact same code is provided by the field_extrawidgets module..

@amateescu, the key area this is different is that it provides a back door into field_ui_default_value_widget() and so a default value may be set via the field edit UI.

Added short summary

I got excited reading through these comments, but now that I've reached the end it looks like this never made it through?

So far the best solution I have to this readonly field issue for D7 is to combine the 2 modules "Field Permissions"(https://www.drupal.org/project/field_permissions) and "Field ReadOnly" (https://www.drupal.org/project/field_readonly)

Anyone else have a different solution they are using?

Shouldn't status be "fixed"? Or am I missing something?

The latest patches are just backporting the patch to D7, but we should open a separate issue for that.

I can't find this in D8 core 8.5.x.

Does I miss something or is lost or removed since 8.5.x? For example I can't find field_field_widget_info_alter in drupal:field core module.

@IT-Cru, see #93 / #2060705: Remove the 'hidden widget' plugin, it is actually never used :)