Spin-off from #2577963: Let entity_ref Selection handlers be in charge of the field validation

Problem/Motivation

EntityReferenceItem does:

  public static function defaultFieldSettings() {
    return array(
      'handler' => 'default:' . (\Drupal::moduleHandler()->moduleExists('node') ? 'node' : 'user'),

So, for base fields :
base e_r fieds that don't bother specifying a 'handler' setting, get a handler for a completely unrelated entity type (user or node, depending on what modules are enabled). That looks potentially very problematic.
SelectionPluginManager::getInstance() already handles "if 'default' then 'default:$target_entity_type", so it seems the default value for 'handler' setting should just be 'default', without trying to guess an arbitrary derivative ?

For configurable fields :
entity_reference_field_config_presave() takes care of saving the correct derivative for the target_type (that is required for config schema, as was discussed in #2436835: Unable to create config schema for entity type specific entity reference selection plugin.), but it only acts on entity_ref' field type, and omits child types (image, file)
Also, a couple of field definitions shipped in standard.profile have the wrong derivative.

Using the wrong derivative for your target type can lead to improper / inconsistent validation, and possibly fatal errors, since the wrong Selection can try to execute EntityQueries with conditions on unknown fields (see #2577963-6: Let entity_ref Selection handlers be in charge of the field validation)

Proposed resolution

Patch :
- Changes the default value for the 'handler' setting to be just 'default', without trying to guess an arbitrary derivative.
- As a proof that this works, lets the node.uid base field use the default value
- Changes entity_reference_field_config_presave() / and entity_reference_field_storage_config_presave() to take care of entity_ref + subtypes, and moves them to field.module since they affect fields that cn be created without entity_ref module
- Adds a hook_post_update() (with a test) to fix existing fields

API changes

None, but base e_r fields that just want to use the default selection plugin for their target type can now omit to specify the 'handler' setting.

Data model changes

None

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug : some e_r/file/image fields can use the wrong Selection handler for their "set of allowed references"
Issue priority Major: This leads to improper validation, and possibly fatal errors

Comments

yched created an issue. See original summary.

yched’s picture

Title: Base e_r fields get the wrong Selection handler if they omit the 'hanbler' setting » Base e_r fields get the wrong Selection handler if they omit the 'handler' setting
Status: Active » Needs review
StatusFileSize
new1.32 KB

Patch, let's see what the bot says.

Status: Needs review » Needs work

The last submitted patch, 2: 2578249_ER_default_handler-1.patch, failed testing.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new2.12 KB
new812 bytes

Migrate test needs adjustment

amateescu’s picture

Component: field system » entity_reference.module
Status: Needs review » Reviewed & tested by the community

Damn, the aspect of base fields was overlooked in #2436835: Unable to create config schema for entity type specific entity reference selection plugin. :(

Since entity_reference_field_config_presave() already handles configurable fields properly and puts the full selection plugin ID (e.g. 'default:taxonomy_term') in config, the patch looks correct.

yched’s picture

yched’s picture

Hm - looks like a few configurable fields shipped in standard.profile have wonky selection plugins :
field.field.node.article.field_image.yml : default:node
field.field.user.user.user_picture.yml : default:node

@amateescu : we should change those to default:file, right ?
I guess that means an update function as well :-/

yched’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +D8 upgrade path

tagging accordingly then

amateescu’s picture

Ugh, yes, we should update them. Basically just copy the code from entity_reference_field_config_presave() into an update function.

The last submitted patch, 2: 2578249_ER_default_handler-1.patch, failed testing.

yched’s picture

@amateescu : Hm - but we're not supposed to call APIs in update funcs, so we can't really rely on $selection_manager->getPluginId() as entity_reference_field_config_presave() does ?

Not sure what the logic should be in that update, actually :-)

yched’s picture

StatusFileSize
new8.11 KB
new5.99 KB

Also, entity_reference_field_config_presave() :
- is actually not doing enough, since it only cares about fields of type 'entity_reference', while the handler needs to be adjusted for "child" field types as well (file, image...)
- should be in field_module rather than entity_ref.module, since this affects fields that can be created without e_r.module
- entity_reference_field_storage_config_presave() should have the same treatment, for consistency

Updated patch moves them to field.module, and makes them handle child types.

Haven't really figured out the question of the update for currently faulty fields.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new11.87 KB
new4.9 KB

Actually, for the update, a hook_post_update() that just resaves the field definitions should work. Added a test as well.

Hopefully this should be committable.

yched’s picture

Title: Base e_r fields get the wrong Selection handler if they omit the 'handler' setting » Some e_r fields get the wrong Selection handler
Issue summary: View changes

Updated the IS and added a beta evaluation

yched’s picture

Issue tags: -beta target +rc target

Status: Needs review » Needs work

The last submitted patch, 13: 2578249_ER_default_handler-13.patch, failed testing.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new12.97 KB
new1.74 KB

Of course it would help if I actually fixed the broken yamls in standard.module...

This should be green.

Status: Needs review » Needs work

The last submitted patch, 18: 2578249_ER_default_handler-18.patch, failed testing.

yched’s picture

[edit : removed uninteresting debug rambling, fail was just me being über-silly]

The last submitted patch, 13: 2578249_ER_default_handler-13.patch, failed testing.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new12.73 KB
new580 bytes

Oh gee. I had left two copies of the logic in that resets 'handler_settings' in field_field_storage_config_update().

/me slams his head against the wall, and goes out for a walk

This should really be green now...

jibran’s picture

Status: Needs review » Needs work
+++ b/core/modules/field/field.install
@@ -26,3 +28,20 @@ function field_update_8001() {
+function field_post_update_entity_reference_handler_setting() {

It is in the wrong file.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new12.66 KB
new2.27 KB

Indeed, I wasn't aware of the [module].post_update.php convention. Thanks !

The last submitted patch, 18: 2578249_ER_default_handler-18.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 24: 2578249_ER_default_handler-24.patch, failed testing.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new12.7 KB
new375 bytes

Yeah, not my day.

Status: Needs review » Needs work

The last submitted patch, 27: 2578249_ER_default_handler-27.patch, failed testing.

yched’s picture

Status: Needs work » Needs review
StatusFileSize
new12.65 KB
new873 bytes

Oh, and UpdatePostUpdateTest, you suck.

Status: Needs review » Needs work

The last submitted patch, 29: 2578249_ER_default_handler-29.patch, failed testing.

yched’s picture

Not sure what to believe between https://qa.drupal.org/pifr/test/1169638 reporting fails and https://dispatcher.drupalci.org/job/default/19890/testReport/Installer/D... reporting passes.

Passes locally, so, retest ?

yched’s picture

Status: Needs work » Needs review

The last submitted patch, 12: 2578249_ER_default_handler-12.patch, failed testing.

jibran’s picture

I think we should add some custom fields in update test to check that it is working for custom fields(not defined by config files) as well and some asserts in EntityUnitTest after saving the ER field config instance so that we can make sure it'll never happen again as in set default handler and assert that it stores default:reference_entity_type . What do you think about it?

  1. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldType/EntityReferenceItem.php
    @@ -53,7 +53,7 @@ public static function defaultStorageSettings() {
    -      'handler' => 'default:' . (\Drupal::moduleHandler()->moduleExists('node') ? 'node' : 'user'),
    +      'handler' => 'default',
    

    +1 to this.

  2. +++ b/core/modules/field/field.module
    @@ -280,3 +282,60 @@ function field_form_config_admin_import_form_alter(&$form, FormStateInterface $f
    +  list($current_handler) = explode(':', $field->getSetting('handler'), 2);
    +  $field->setSetting('handler', $selection_manager->getPluginId($target_type, $current_handler));
    

    I know this is c/p of existing code but can we please add comments to explain this.

  3. +++ b/core/modules/field/src/Tests/Update/EntityReferenceHandlerSettingUpdateTest.php
    @@ -0,0 +1,52 @@
    +    $this->assertEqual($settings['handler'], 'default:node');
    

    :(

  4. +++ b/core/modules/field/src/Tests/Update/EntityReferenceHandlerSettingUpdateTest.php
    @@ -0,0 +1,52 @@
    +    $this->assertEqual($settings['handler'], 'default:file');
    

    :)

The last submitted patch, 22: 2578249_ER_default_handler-22.patch, failed testing.

The last submitted patch, 24: 2578249_ER_default_handler-24.patch, failed testing.

The last submitted patch, 27: 2578249_ER_default_handler-27.patch, failed testing.

The last submitted patch, 12: 2578249_ER_default_handler-12.patch, failed testing.

yched’s picture

Issue summary: View changes
StatusFileSize
new3.01 KB
new15.96 KB
new4.29 KB

@jibran #35 :

I think we should add some custom fields in update test to check that it is working for custom fields(not defined by config files)

I don't see the point. Fields shipped in config are just regular configurable fields, no different from the ones created in the UI. The test tests a configurable field that's present in drupal-8.bare.standard.php.gz, whether it came from the install profile or was added later on makes no difference.

as well and some asserts in EntityUnitTest after saving the ER field config instance so that we can make sure it'll never happen again as in set default handler and assert that it stores default:reference_entity_type

That is actually already tested in EntityReferenceItemTest::testSelectionHandlerSettings(), because entity_reference_field_config_presave() took care of it.

The thing that was broken was that entity_reference_field_config_presave() didn't take care of sub field types (file, image), and field_field_config_presave() now does. Added a test for that in FileItemTest (fails on HEAD)

Also, added a comment for the correct but non-explicit flow for "reassign the correct handler plugin in the fields when the target_type changes in the field storage". Added more explicit tests in EntityReferenceItemTest::testSelectionHandlerSettings() (they pass on HEAD)

#35.1 :-)
#35.2 Added a comment
#35.3 .4 :-)

xjm’s picture

Issue tags: -rc target

See https://groups.drupal.org/node/484788 for more information on the rc target tag. This issue should possibly be retagged as rc deadline instead?

yched’s picture

@xjm : I think rc target is fine. This breaks no APIs and can happen after RC1, but would be really best to get in before RC1 because the current behavior (e.g. field_image using the NodeSelection plugin) is very weird and can lead to nasty consequences.

The last submitted patch, 40: 2578249_ER_default_handler-test_only.patch, failed testing.

yched’s picture

Test-only patch fails, patch passes...

RTBC anyone ? ;-)

The last submitted patch, 40: 2578249_ER_default_handler-test_only.patch, failed testing.

jibran’s picture

Status: Needs review » Reviewed & tested by the community

Here we go thanks for the explanation and fixes.

yched’s picture

StatusFileSize
new15.96 KB

Reroll

yched’s picture

StatusFileSize
new69.34 KB

Reroll after #2571533: Allow setting custom storage on FieldStorageConfig, that also added a field_post_update

yched’s picture

StatusFileSize
new15.98 KB

As mentioned by @alexpott on IRC, #48 was too big, diffed against the wrong HEAD, I guess.

Re-rebased, sizes match.

yched’s picture

Status: Reviewed & tested by the community » Needs work

Reroll needed after #2520540: Enforced configuration dependencies shouldn't have to be repeated in the calculated dependencies, that had to adjust the list of post_updates in UpdatePostUpdateTest as well.

Not where I can do this atm, anyone up for it ? ;-)

fabianx’s picture

Issue tags: +Needs reroll
yched’s picture

Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs reroll
StatusFileSize
new15.97 KB

Reroll

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 52: 2578249_ER_default_handler-52.patch, failed testing.

Status: Needs work » Needs review
yched’s picture

Status: Needs review » Reviewed & tested by the community

Bot fluke, back to green

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

This issue has test coverage and a test upgrade path. Committed 60b75a2 and pushed to 8.0.x. Thanks!

+++ b/core/profiles/standard/config/install/field.field.node.article.field_image.yml
@@ -32,6 +32,6 @@ settings:
-  handler: 'default:node'
+  handler: 'default:file'

+++ b/core/profiles/standard/config/install/field.field.user.user.user_picture.yml
@@ -32,6 +32,6 @@ settings:
-  handler: 'default:node'
+  handler: 'default:file'

Yeah this is super confusing and wrong.

diff --git a/core/modules/entity_reference/entity_reference.module b/core/modules/entity_reference/entity_reference.module
index 6d27e56..6cbea17 100644
--- a/core/modules/entity_reference/entity_reference.module
+++ b/core/modules/entity_reference/entity_reference.module
@@ -11,7 +11,6 @@
 use Drupal\Core\Render\Element;
 use Drupal\Core\Routing\RouteMatchInterface;
 use Drupal\field\Entity\FieldConfig;
-use Drupal\field\FieldStorageConfigInterface;
 
 /**
  * Implements hook_help().

Unused use fixed on commit.

  • alexpott committed 60b75a2 on 8.0.x
    Issue #2578249 by yched: Some e_r fields get the wrong Selection handler
    
larowlan’s picture

This causes major issues for DER, because there is no 'handler' setting - but field_field_config_pre_save expects it to be.

We should be checking default settings to see if such a key exists or checking schema. Feels dirty that field.module blindly expects there to be a handler setting.

Will get around it with adding a fake value and then removing it in a presave of our own.

yched’s picture

@larowlan : Yes, there's kind of an expectation that if a field type extends another one, it has all the settings of the parent. Otherwise things might break any time the parent class adds some code using $settings['the_setting'] in a point release ?

larowlan’s picture

We got around it with #2581609: field_field_config_presave expects there to be a 'handler' setting but there isn't - hook_module_implements_alter to nix the field.module one in the case of DER

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.