Problem/Motivation

I opened #3134245: Run drush imports as user=1 to solve part of the problem. But that solution is very much a sledgehammer solution when perhaps a more valid solution is to just switch the user if entity validation is enabled. This does the latter.

Proposed resolution

Switch to the entity owner if it implements EntityOwnerInterface

Test the latest patch by enabling validation on the destination entity i.e. validate: true.

source:
  plugin: d7_node
  node_type: page
process:
  title: title
  type: type
  body: body
destination:
  plugin: 'entity:node'
  validate: true

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#85 diff-78-85.txt1.68 KBmikelutz
#85 3134470-85.drupal.patch23.63 KBmikelutz
#78 3134470-78.patch23.59 KBgabesullice
#78 interdiff.txt2.72 KBgabesullice
#73 3134470-73.patch23.08 KBgabesullice
#73 3134470-73-FAIL.patch12.2 KBgabesullice
#73 interdiff.txt2.3 KBgabesullice
#72 3134470-72.patch23.09 KBsudiptadas19
#71 interdiff_70-71.txt1 KBsudiptadas19
#71 3134470-71.patch23.1 KBsudiptadas19
#70 interdiff_64-70.txt5.14 KBsudiptadas19
#70 3134470-70.patch23.06 KBsudiptadas19
#64 3134470-63.patch23.04 KBheddn
#63 interdiff_61-63.txt1.14 KBheddn
#63 interdiff_61-63.txt1.14 KBheddn
#61 interdiff_48-61.txt2.98 KBheddn
#61 3134470-61.patch23.03 KBheddn
#48 3134470-48.patch23.04 KBheddn
#48 interdiff_42-48.txt10.54 KBheddn
#46 3134470-46.patch22.79 KBheddn
#46 interdiff_44-46.txt6.41 KBheddn
#44 3134470-44.patch16.38 KBheddn
#44 interdiff_42-44.txt957 bytesheddn
#42 interdiff_40-42.txt1.37 KBheddn
#42 3134470-42.patch16.29 KBheddn
#40 3134470-40.patch16.78 KBheddn
#40 interdiff_38-40.txt1.04 KBheddn
#38 3134470-36.patch16.61 KBheddn
#36 interdiff_33-36.txt9.29 KBheddn
#33 interdiff_30-33.txt768 bytesridhimaabrol24
#33 3134470-33.patch8.21 KBridhimaabrol24
#30 interdiff_26-30.txt2.58 KBridhimaabrol24
#30 3134470-30.patch8.23 KBridhimaabrol24
#26 3134470-26.patch8.26 KBheddn
#26 interdiff_22-26.txt3.67 KBheddn
#22 interdiff.txt3.06 KBquietone
#22 3134470-22.patch8.24 KBquietone
#22 3134470-22-fail.patch5 KBquietone
#16 3134470-16.patch8.24 KBheddn
#16 interdiff_11-16.txt3.01 KBheddn
#11 3134470-11.patch7.76 KBheddn
#11 3134470-test_only.patch4.53 KBheddn
#2 3134470.patch3.23 KBheddn
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

heddn created an issue. See original summary.

heddn’s picture

Status: Active » Needs review
FileSize
3.23 KB
heddn’s picture

Kristen Pol’s picture

Assigned: Unassigned » Kristen Pol

Assigning to myself for review.

Kristen Pol’s picture

From May 7th:

Thanks for the patch.

1) Reviewed the code and it looks straight forward. I compared it to the AccountSwitcherInterfaceusage in core/lib/Drupal/Core/Cron.php and see they are doing dependency injection in the constructor but this is adding it via create. I'd need to look at more examples which I can't do at the moment (getting pulled away). I imagine the code is perfectly fine though. :)

2) Patch applies cleanly:

[mac:kristen:drupal-9.1.x-dev]$ patch -p1 < 3134470.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php

3) What is the best way to manually test this?

==

UPDATED (May 8th): I searched the codebase for something like:

protected function getX(): YInterface {

and didn't see anything. I like to review existing code that's similar when reviewing to help me validate the approach.

Kristen Pol’s picture

Assigned: Kristen Pol » Unassigned
Kristen Pol’s picture

Ok, I was being too hasty yesterday and now see that this code is using some of the same code from the referenced migrate_tools module patch in #3134470: Switch to entity owner in EntityContentBase during validation which was based on the default_content module.

Looking at this code again and at other migrate code, unfortunately, it's not clear to me why accountSwitcher isn't part of the constructor and the getAccountSwitcher function is used.

heddn’s picture

So, the reason I didn't do constructor injection is that while constructors are not technically part of BC, there are several cases where contrib and custom implementations extend the constructor and add their own additional arguments. If we did that here, we'd break lots of people. The solution I used still uses injection, but is not going to break many thousand of sites.

Kristen Pol’s picture

Thanks for the explanation. Now I have dumb question...

Can't it not be added to the parameters in the constructor but still initialized in the constructor (presumably if people are extending the class they are calling the parent as this is a best practice, no?) and then the function wouldn't be required as it would always be set? Or we can't assume they would call the parent constructor?

heddn’s picture

Manually test... build out a source that sets the format on a text field. Make sure the format is one that requires elevated access. Additionally, make sure validation: true on the destination. Then run the migration via drush. Drush always runs as anonymous. Without the patch, validation will fail. With the patch, validation will succeed if the user associated with the entity has access to add the format.

    $definition = [
      'source' => [
        'plugin' => 'embedded_data',
        'data_rows' => [
          [
            'id' => 1,
            'uid' => 1,
            'body' => [
              'value' => 'foo',
              'format' => 'full_html',
            ]
          ],
          [
            'id' => 2,
            'uid' => 0,
            'body' => [
              'value' => 'bar',
              'format' => 'full_html',
            ]
          ],
        ],
        'ids' => [
          'id' => ['type' => 'integer'],
        ],
      ],
      'process' => [
        'id' => 'id',
        'body/value' => 'body/value',
        'body/format' => 'body/format',
      ],
      'destination' => [
        'plugin' => 'entity:node:page',
        'validation' => TRUE,
      ],
    ];
heddn’s picture

Here we add some tests. Interdiff is the test only patch.

heddn’s picture

Self review, to pick up on next re-roll:

--- a/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
+++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
@@ -168,6 +168,7 @@ public function testAccountSwitcher() {
     $role = Role::create([
       'id' => 'admin',
       'label' => 'admin',
+      'is_admin' => TRUE,
     ]);
     $role->grantPermission($filter_test_format->getPermissionName());
     $role->save();
@@ -234,7 +235,7 @@ public function testAccountSwitcher() {
     $this->container->get('current_user')->setAccount($normal_user);
     $this->runImport($definition);
 
-    $this->assertSame(sprintf('2: [entity_test: 2]: user_id.0.target_id=This entity (<em class="placeholder">user</em>: <em class="placeholder">%s</em>) cannot be referenced.||%s.0.format=The value you selected is not a valid choice.', $normal_user->id(), $field_name), $this->messages[0], 'First message should have 3 validation errors.');
+    $this->assertSame(sprintf('2: [entity_test: 2]: user_id.0.target_id=This entity (<em class="placeholder">user</em>: <em class="placeholder">%s</em>) cannot be referenced.||%s.0.format=The value you selected is not a valid choice.', $normal_user->id(), $field_name), $this->messages[0], 'First message should have 2 validation errors.');
     $this->assertArrayNotHasKey(1, $this->messages, 'Second message should not exist.');
   }
heddn’s picture

re #9: can we add the injection in constructor (not create) function?

We could move it over, but I prefer not to call \Drupal::service() unless I need to. The constructor doesn't have access to the container. While create does have access. And adding the getter lets me easily stub out the service in tests if I ever need to do that.

Kristen Pol’s picture

Status: Needs review » Needs work

Thanks for the test.

1) Patch applied cleanly:

[mac:kristen:drupal-9.1.x-dev]$ patch -p1 < 3134470-11.patch 
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
Hunk #3 succeeded at 47 with fuzz 1.

2) Reviewed the code and could follow most of it though it would be nice to have some comments :) though there aren't many in the other tests.

  1. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -33,9 +47,12 @@ class MigrateEntityContentValidationTest extends KernelTestBase {
    -    $this->installConfig(['system', 'user']);
    

    Unclear why this was deleted though I assume it wasn't needed.

  2. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -33,9 +47,12 @@ class MigrateEntityContentValidationTest extends KernelTestBase {
    +
     
    

    Nitpick: extra new line.

  3. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +157,87 @@ public function test2() {
    +  /**
    +   * Tests validation where entity owner becomes the user who saves the entity.
    +   */
    +  public function testAccountSwitcher() {
    

    IMO this comment isn't clear. Not sure what's better but maybe something along the lines of the following, unless I'm not understanding the test:

    Tests validation where user without filter format permission saves content.

3) Still have question from #9.

4) Moving back to "Needs work" in case any of 2) should be addressed.

Kristen Pol’s picture

We crossposted so #14.3 was answered in #13. Thanks!

heddn’s picture

Status: Needs work » Needs review
FileSize
3.01 KB
8.24 KB

Adds comments to the test to respond to most of the feedback.

#9: see #13
#14.1: this is not removed but became $this->installConfig(['field', 'filter_test', 'system', 'user']);
#14.2: fixed
#14.3: fixed

Kristen Pol’s picture

Status: Needs review » Needs work

Thanks for the updates.

1) Reviewed the interdiff and the changes cover the items noted in #14.

The comments help a lot! :)

+++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
@@ -33,9 +48,11 @@ class MigrateEntityContentValidationTest extends KernelTestBase {
+    $this->installSchema('system', ['sequences']);
+    $this->installConfig(['field', 'filter_test', 'system', 'user']);

Ah, see that installConfig was moved after installSchema. Thanks for pointing that out.

2) Patch applies to 8.9 (with offsets), 9.0, and 9.1.

[mac:kristen:drupal-8.9.x-dev]$ patch -p1 < 3134470-16.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
Hunk #1 succeeded at 9 (offset 1 line).
Hunk #2 succeeded at 19 (offset 1 line).
Hunk #3 succeeded at 111 (offset 7 lines).
Hunk #4 succeeded at 149 (offset 7 lines).
Hunk #5 succeeded at 159 (offset 7 lines).
Hunk #6 succeeded at 202 (offset 7 lines).
Hunk #7 succeeded at 403 (offset 7 lines).
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
Hunk #3 succeeded at 48 with fuzz 1.
[mac:kristen:drupal-8.9.x-dev]$ cd90
[mac:kristen:drupal-9.0.x-dev]$ patch -p1 < 3134470-16.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
[mac:kristen:drupal-9.0.x-dev]$ cd91
[mac:kristen:drupal-9.1.x-dev]$ patch -p1 < 3134470-16.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php

3) Reviewed the patch again. Only a couple nitpicks I missed before caught my eye. Not sure they need to be changed but I'll move back to needs work in case they should be.

  1. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -184,8 +195,18 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
    +    if (($entity instanceof EntityOwnerInterface) && $user = $entity->getOwner()) {
    

    Sorry, should have caught this before.

    Nitpick: over 80 characters.

  2. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -184,8 +195,18 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
    +    if (($entity instanceof EntityOwnerInterface) && $user = $entity->getOwner()) {
    

    Same as above.

4) Not sure this will need manual testing before this can considered for RTBC.

heddn’s picture

Status: Needs work » Needs review

80 char limit is for comments, not code. And it would be caught by PHPCS if it were a concern for the testbot. Switching back to NR for any other feedback.

Kristen Pol’s picture

Sorry, lack of sleep is becoming apparent here! Ignore me ;)

@heddn In your opinion, does this need manual testing prior to RTBC? Thanks.

heddn’s picture

re manual testing: I don't think so.

Kristen Pol’s picture

Status: Needs review » Reviewed & tested by the community

Considering all the above, marking RTBC. Thanks for your patience!

quietone’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
5 KB
8.24 KB
3.06 KB

I'd really like to see a failing patch and I decided to make it myself. I started from the patch in #16 and made a failing patch and re-uploaded 16.

And then I found I have some questions.

  1. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -184,8 +195,18 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
    +    if (($entity instanceof EntityOwnerInterface) && $user = $entity->getOwner()) {
    

    This test is done twice in the same method. Can't we do it just once? Should we check if the entity owner is the same as the current user before switching and switching back?

  2. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -375,4 +396,17 @@ public function getHighestId() {
    +   * Get the account switcher service.
    

    Should be 'Gets'.

  3. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +157,95 @@ public function test2() {
    +    // Filter formats have permissions tied to user/user roles.
    

    Does this mean user and user roles?

  4. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +157,95 @@ public function test2() {
    +            ]
    ...
    +            ]
    

    Missing trailing ','

  5. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +157,95 @@ public function test2() {
    +    // The second, non-permissioned user import should fail validation.
    

    I must be reading this wrong. The user is set to $normal_user but the content type that is owned by $normal user fails to migrate?

The last submitted patch, 22: 3134470-22-fail.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Kristen Pol’s picture

Nice! Glad this is getting more eyes on it :)

quietone’s picture

Status: Needs review » Needs work

Setting to NW for #22

heddn’s picture

Status: Needs work » Needs review
FileSize
3.67 KB
8.26 KB

re #22.5:
The fail test will fail on entity 1, because it isn't switched to user 1 (admin user) and it attempts to run validation as an anonymous user. This user doesn't have the needed access.

But then the test goes further to assert that entity 2 will fail, because a non privileged user (user 2) doesn't have access to use "filter_test" filter. Both test cases are needed.

Fixes other feedback from #22, points 1-4. I'm not sure I like the combined check any more then what was there before, but it does only execute things once.
$switch_user = (($entity instanceof EntityOwnerInterface) && $user = $entity->getOwner());

Kristen Pol’s picture

Thanks for the update.

1) Reviewing the interdiff in #26 and the items in #22, it appears that they are all addressed. I noticed one missing Oxford comma but I saw this so I don't want to hold this up further:

https://www.drupal.org/drupalorg/style-guide/words-phrases

But it's not doctrine. No one will be shamed for omitting an Oxford comma. (Editors will probably add that for you anyway.) And it doesn't create any grammar robot overlords (we hope).

  1. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -161,7 +160,7 @@ public function test2() {
    +    // Filter formats have permissions tied to user, user roles and permissions.
    

    If this patch ends up being updated again, add Oxford comma after "roles".

  2. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -241,7 +240,8 @@ public function testEntityOwnerValidation() {
    -    // The second, non-permissioned user import should fail validation.
    +    // The second, non-permissioned user import should fail validation because
    +    // they do not have access to use "filter_test" filter.
    

    Reading again, although I understand what "non-permissioned" means here, it turns out it's not commonly used. Only 4,800 results in Google for that. It's not used in core anywhere. Probably want to reword that. Perhaps just remove it since IMO the expanded text covers it.

    The second user import should fail validation because they do not have access to use "filter_test" filter.

2) Patch applies cleanly to 8.9, 9.0, and 9.1.

[mac:kristen:drupal-8.9.x-dev]$ patch -p1 < 3134470-26.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
Hunk #1 succeeded at 9 (offset 1 line).
Hunk #2 succeeded at 19 (offset 1 line).
Hunk #3 succeeded at 111 (offset 7 lines).
Hunk #4 succeeded at 149 (offset 7 lines).
Hunk #5 succeeded at 159 (offset 7 lines).
Hunk #6 succeeded at 202 (offset 7 lines).
Hunk #7 succeeded at 404 (offset 7 lines).
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
Hunk #3 succeeded at 47 with fuzz 1.
[mac:kristen:drupal-8.9.x-dev]$ cd90
[mac:kristen:drupal-9.0.x-dev]$ patch -p1 < 3134470-26.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
[mac:kristen:drupal-9.0.x-dev]$ cd91
[mac:kristen:drupal-9.1.x-dev]$ patch -p1 < 3134470-26.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
Kristen Pol’s picture

Status: Needs review » Needs work

Moving to "Needs work" #27 as I think the rewording in 27.1.2 would make it easier to read.

longwave’s picture

+++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
@@ -184,8 +195,19 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
+    $switch_user = (($entity instanceof EntityOwnerInterface) && $user = $entity->getOwner());
+    if ($switch_user) {
+      $this->getAccountSwitcher()->switchTo($user);
+    }

Instead of having two variables here, $switch_user as a boolean flag and then $user as the object, why not just use $user as both the flag and the object?

$user = $entity instanceof EntityOwnerInterface ? $entity->getOwner() : NULL;
if ($user) {
  ...
}

Also I think we tend to use $account for generic account objects and $user as the actual "current user" though not sure this is set in stone, not sure if that would make it clearer here or not.

ridhimaabrol24’s picture

ridhimaabrol24’s picture

Status: Needs work » Needs review
heddn’s picture

Status: Needs review » Needs work
+++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
@@ -140,6 +156,96 @@ public function test2() {
+    // Filter formats have permissions tied to user, user roles, and permissions.

Nit: this is now 81 chars. How about (53 chars): "Filter format access is impacted by user permissions."

ridhimaabrol24’s picture

Status: Needs work » Needs review
FileSize
8.21 KB
768 bytes

Altered the comment as suggested in #32.

Kristen Pol’s picture

Status: Needs review » Reviewed & tested by the community

Thanks for the update.

1) Reviewed the interdiff in #33 and see it addresses item from #32.

2) Patch applies cleanly to 9.1.

[mac:kristen:drupal-9.1.x-dev]$ patch -p1 < 3134470-33.patch
patching file core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
patching file core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php

3) Tests pass.

4) Marking RTBC based on this review and the comments above.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

@heddn if they override the constructor then they'll be calling parent::__construct() - if we add AccountSwitcher to our __construct as $account_swticher = NULL and then trigger a deprecation when called without it we're fine and we'll break no one.

This pattern of doing pseudo setter inject in ::create() methods is okay for custom or contrib where you want to insulate yourself from potentially breaking upstream changes but I don't think we should use it all the time in core.

heddn’s picture

Status: Needs work » Needs review
FileSize
9.29 KB

I'm not sure if we should re-order the arguments on EntityComment, EntityRevision and EntityUser. But this picks up the rest of the requested changes.

Kristen Pol’s picture

@heddn There's only an interdiff. Did you want to post the patch too or are you looking for feedback on the changes first?

heddn’s picture

Status: Needs review » Needs work

The last submitted patch, 38: 3134470-36.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
1.04 KB
16.78 KB
alexpott’s picture

Yeah we shouldn't re-order the arguments. In an ideal world the comment and user classes would be final and their constructor marked as @internal so we can change the order. But that's not the world we're in.

  1. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -375,4 +403,17 @@ public function getHighestId() {
    +  /**
    +   * Gets the account switcher service.
    +   *
    +   * @return \Drupal\Core\Session\AccountSwitcherInterface
    +   *   The account switcher service.
    +   */
    +  protected function getAccountSwitcher(): AccountSwitcherInterface {
    +    if (!isset($this->accountSwitcher)) {
    +      $this->accountSwitcher = \Drupal::service('account_switcher');
    +    }
    +    return $this->accountSwitcher;
    +  }
    

    This is no longer needed.

  2. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -184,8 +201,19 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
    +    // Entity validation can require the user that owns the entity. Switch to
    +    // use that user during validation.
    +    $account = $entity instanceof EntityOwnerInterface ? $entity->getOwner() : NULL;
    

    Do we have to be concerned about migration dependencies. Like is the user guaranteed to exist at this point?

  3. +++ b/core/modules/user/src/Plugin/migrate/destination/EntityUser.php
    @@ -95,9 +96,15 @@ class EntityUser extends EntityContentBase {
    -  public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, EntityStorageInterface $storage, array $bundles, EntityFieldManagerInterface $entity_field_manager, FieldTypePluginManagerInterface $field_type_manager, PasswordInterface $password) {
    -    parent::__construct($configuration, $plugin_id, $plugin_definition, $migration, $storage, $bundles, $entity_field_manager, $field_type_manager);
    +  public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, EntityStorageInterface $storage, array $bundles, EntityFieldManagerInterface $entity_field_manager, FieldTypePluginManagerInterface $field_type_manager, PasswordInterface $password, AccountSwitcherInterface $account_switcher = NULL) {
    +    if ($account_switcher === NULL) {
    

    I was pondering about this from a migrate dependencies perspective and then realised that actually injecting this service is pointless because the user entity is not a EntityOwnerInterface entity.

    Kinda amusing to have to make this change for something that for this object will never be used.

heddn’s picture

FileSize
16.29 KB
1.37 KB

Re #41.2: yes we do have to concern ourselves with this. Ergo the check for account in the next line.

alexpott’s picture

@heddn re #41.2 probably worth adding a comment in the code about that then.

heddn’s picture

FileSize
957 bytes
16.38 KB

Status: Needs review » Needs work

The last submitted patch, 44: 3134470-44.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
6.41 KB
22.79 KB
quietone’s picture

Status: Needs review » Needs work

A brief look at the recent changes, since the RTBC.

+++ b/core/modules/comment/src/Plugin/migrate/destination/EntityComment.php
@@ -53,9 +54,15 @@ class EntityComment extends EntityContentBase {
+      @trigger_error('The account_switcher service must be passed to ' . __NAMESPACE__ . '\EntityComment::__construct(). It was added in drupal:9.1.0 and will be required before drupal:10.0.0.', E_USER_DEPRECATED);

If I am reading the policy correctly this needs to have when it will be removed and a CR.

%thing% is deprecated in %deprecation-version% (free text describing what will happen) %removal-version%. %extra-info%(optional). See %cr-link%

+++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityRevision.php
@@ -114,11 +115,11 @@ class EntityRevision extends EntityContentBase {
+  public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, EntityStorageInterface $storage, array $bundles, EntityFieldManagerInterface $entity_field_manager, FieldTypePluginManagerInterface $field_type_manager, AccountSwitcherInterface $account_switcher) {

Doesn't this need a trigger_error as well?

heddn’s picture

Status: Needs work » Needs review
FileSize
10.54 KB
23.04 KB

CR added and linked and deprecation test added.

#47.2: I don't think it is needed. In fact, I removed it from a few other places just now. The parent class will throw the deprecation.

alexpott’s picture

I'm wondering if this behaviour should be optional. For example, imagine permissions are being changed as part of the migration. Also maybe for performance reasons or other reasons someone if running the migration as user 1.

heddn’s picture

They can always not activate validation. But if you want entity validation, then you need to use the right user ID.

heddn’s picture

BTW, the only way to run (via drush) as user 1 is via the account switcher. For a very long time, these migrations have been run as anonymous, not even the admin user. This would be different via migrate_drupal_ui, but then you really aren't so worried about performance or anything at that point, because entity validation (currently) isn't even enabled for that UI.

quietone’s picture

I read the CR and it looks fine to me and does the patch. I don't see anything still to do.

However, I have one question. I applied that patch and ran phpcs on migrate/destination/EntityContentBase.php and get the following error and there are not coding standard errors reported by drupalci. Does the message need to change?

----------------------------------------------------------------------
 141 | ERROR   | The trigger_error message 'The account_switcher
     |         | service must be passed to  __NAMESPACE__
     |         | \EntityContentBase::__construct(). It was added in
     |         | drupal:9.1.0 and will be required before
     |         | drupal:10.0.0. See
     |         | https://www.drupal.org/node/3142975.' does not match
     |         | the relaxed standard format: %thing% is deprecated
     |         | in %deprecation-version% any free text
     |         | %removal-version%. %extra-info%. See %cr-link%
alexpott’s picture

But if you want entity validation, then you need to use the right user ID.

Is this the case? We don't switch before saving a node that was created by someone else.

heddn’s picture

I'm not exactly sure what to suggest then. In the case where someone is saving content that is owned by another user, we do run validation. And if the user doesn't have access to full html in the body field, then the save will fail. We could always run it as root, but that seems like cheating and it would hide valid validation errors.

alexpott’s picture

@heddn yeah I think this is a tricky thing. It's going to depend so much on how your new site is configured as to what is the correct thing to do. I'm not sure it is possible for there to be a right way.

heddn’s picture

re #53: usually user migrations are supposed to happen before all other migrations. For just that reason. If someone has entity validation configured for a migration, then they want to validate the entity in the most valid way. Which seems to me as the entity owner.

From #3134245-7: Run drush imports as user=1, some other use cases that might help guide us:

It fails on entity reference for the ownership of the referenced entity.

A migration which create some file records, and assign them to user 1 using the default_value process plugin
A later migration creates media items where an image field refers to the file...
... entity validation during migration fails with the message: "[media: 341]: field_media_image.0=You do not have access to the referenced entity (file: 341)"

Path alias migrations can fail validation too: "\[path_alias: 12842, revision: 1533\]: path.0.value=Either the path '/node/14' is invalid or you do not have access to it."

The later issue w/ path alias makes me wonder if using user = 1 is the better solution. Instead of the entity owner.

heddn’s picture

Issue summary: View changes
Wim Leers’s picture

  1. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -184,7 +201,18 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
    +    // Entity validation can require the user that owns the entity. Switch to
    +    // use that user during validation.
    

    Can we point to an example validation constraint that requires this? (using @see)

  2. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -184,7 +201,18 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
    +    $account = $entity instanceof EntityOwnerInterface ? $entity->getOwner() : NULL;
    

    🤔 This implies that every migration using a EntityContentBase subclass as a destination for which the entity type implements EntityOwnerInterface MUST have a required migration dependency on the d7_user migration.

    Is it at all possible to add an assert() that adds the logic that checks that?

    Preferably in \Drupal\migrate\Plugin\migrate\destination\EntityContentBase::__construct(), where a MigrationInterface object is readily available, and also to avoid us asserting that for every entity we validate, which is pointless.

  3. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,96 @@ public function test2() {
    +    // Filter format access is impacted by user permissions.
    ...
    +    // Add a "body" field with the filter format.
    

    🤓 s/filter format/text format/

  4. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,96 @@ public function test2() {
    +    // Attempt to migrate entities. One owned by the admin user, the other is a
    +    // non-permissioned user.
    

    🤓 The second sentence here doesn't really make sense.

  5. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,96 @@ public function test2() {
    +    $this->assertSame(sprintf('2: [entity_test: 2]: user_id.0.target_id=This entity (<em class="placeholder">user</em>: <em class="placeholder">%s</em>) cannot be referenced.||%s.0.format=The value you selected is not a valid choice.', $normal_user->id(), $field_name), $this->messages[0], 'First message should have 2 validation errors.');
    

    👎 Not specifying a failure message here (that second parameter to assertSame()) yields far more helpful failing test output.

    Could you please remove that? 🙏😊

Kristen Pol’s picture

Status: Needs review » Needs work

Back to needs work per #58.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

heddn’s picture

Status: Needs work » Needs review
FileSize
23.03 KB
2.98 KB

All feedback from #58 addressed.

For #58.2: I don't think we can make this assumption. We cannot assume all migrations are for drupal upgrades. Folks can also migrate from any source they want for any motive they want.

Status: Needs review » Needs work

The last submitted patch, 61: 3134470-61.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
1.14 KB
1.14 KB
heddn’s picture

FileSize
23.04 KB
Wim Leers’s picture

Status: Needs review » Needs work

#61: RE #58.2: Right, that makes sense! 👍

My key remaining concern is then that we ensure that the test coverage actually tests what we need it to test. @quietone already asked this question and proved it in #22 👍


I would RTBC this … but unfortunately at least the first of the following remarks must be addressed before this can get committed. The remainder are nitpicks that I am fine with not being addressed, since they're only in tests.

  1. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -122,11 +131,18 @@ class EntityContentBase extends Entity implements HighestIdInterface, MigrateVal
    +      @trigger_error('The account_switcher service must be passed to ' . __NAMESPACE__ . '\EntityContentBase::__construct(). It was added in drupal:9.1.0 and will be required before drupal:10.0.0. See https://www.drupal.org/node/3142975.', E_USER_DEPRECATED);
    
    +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentBaseTest.php
    @@ -301,4 +302,24 @@ public function testStubRows() {
    +    $this->expectDeprecation('The account_switcher service must be passed to Drupal\migrate\Plugin\migrate\destination\EntityContentBase::__construct(). It was added in drupal:9.1.0 and will be required before drupal:10.0.0. See https://www.drupal.org/node/3142975.');
    

    This now needs to be updated to 9.2.0.

  2. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,95 @@ public function test2() {
    +    // Filter format access is impacted by user permissions.
    

    Missed this spot in #58.

  3. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,95 @@ public function test2() {
    +    // Create 2 users, an admin user who has filter permission and another who
    

    Related nit: s/who has filter permission/who has permission to use this text format/

quietone’s picture

What about the points raised in #56 about files and path aliases? In that comment heddn suggests that path alias should be owned by uid 1. Will enabling validation on path alias migration break things for people?

I re-read the issue and reviewed the patch, finding a few things, showing that I have been dabbling in coding standards issues. The comments in test, MigrateEntityContentValidationTest::testEntityOwnerValidation have improved since my last look at this and it is much easier to understand. And along with the suggestion by Wim Leers it will be even better.

  1. +++ b/core/modules/comment/src/Plugin/migrate/destination/EntityComment.php
    @@ -53,9 +54,11 @@ class EntityComment extends EntityContentBase {
    +   * @param \Drupal\Core\Session\AccountSwitcherInterface $account_switcher
    

    Null is allowed here. So methinks this should be
    @param \Drupal\Core\Session\AccountSwitcherInterface|null $account_switcher

  2. +++ b/core/modules/user/src/Plugin/migrate/destination/EntityUser.php
    @@ -95,9 +96,11 @@ class EntityUser extends EntityContentBase {
    +   * @param \Drupal\Core\Session\AccountSwitcherInterface $account_switcher
    

    Same here

Next I read the CR. I do like that it is concise but in this case perhaps too concise. I think that the phrase , "When validation occurs during a migration" needs explanation. I suggesting adding an example, as in the IS here that shows how one enables validation. And even a link to the CR that added validation would be nice, https://www.drupal.org/node/3073707.

heddn’s picture

Issue tags: +Novice

Mostly updates to comments that I'd consider novice issues.

quietone’s picture

I remembered there was a tag for updating the change record. See #66

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

sudiptadas19’s picture

Status: Needs work » Needs review
FileSize
23.06 KB
5.14 KB

All the feedback from #65 and #66 addressed.

sudiptadas19’s picture

sudiptadas19’s picture

Fixed PHPCBF whitespace issues.

gabesullice’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Novice
FileSize
2.3 KB
12.2 KB
23.08 KB

Nice! This test case demonstrates exactly the same issue that I encountered myself and this is exactly the solution I was going to propose before I found this issue :D


+++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentBaseTest.php
@@ -301,4 +302,24 @@ public function testStubRows() {
+    $this->expectDeprecation('The account_switcher service must be passed to Drupal\migrate\Plugin\migrate\destination\EntityContentBase::__construct(). It was added in drupal:9.3.0 and will be required before drupal:10.0.0. See https://www.drupal.org/node/3142975.');

Nit: it won't be required before Drupal 10. The phrasing below matches other, similar deprecation examples.

Calling Drupal\migrate\Plugin\migrate\destination\EntityContentBase::__construct() without the $account_switcher argument is deprecated in drupal:9.3.0 and will be required in drupal:10.0.0. See https://www.drupal.org/node/3142975

The overwhelming majority of them do not have a period after the See <url> so I removed that dot as well.

IMHO, this can be RTBC'd because everything else looks good here. I made the change above myself so that this doesn't die by a death of a 1000 nits.

gabesullice’s picture

I've updated the CR

gabesullice’s picture

Category: Task » Bug report

IMO, this is a bug since it is almost impossible to validate imported entities which have a text format that the anonymous user isn't allowed to use (which is often the case)

The last submitted patch, 73: 3134470-73-FAIL.patch, failed testing. View results

larowlan’s picture

Status: Reviewed & tested by the community » Needs review

One question and a couple of nits

  1. +++ b/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php
    @@ -185,7 +202,20 @@ public function isEntityValidationRequired(FieldableEntityInterface $entity) {
         $violations = $entity->validate();
    

    should we wrap this in a try catch and then do the switch back in a finally?

    just in case something in validation throws an exception?

  2. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,99 @@ public function test2() {
    +    /**
    +     * @var \Drupal\filter\FilterFormatInterface $filter_test_format
    +     */
    

    nit; should we use assert($filter_test_format instanceof FilterFormatInterface) instead?

  3. +++ b/core/modules/migrate/tests/src/Kernel/MigrateEntityContentValidationTest.php
    @@ -140,6 +156,99 @@ public function test2() {
    +    /**
    +     * @var \Drupal\user\RoleInterface $role
    +     */
    

    nit: similarly here?

gabesullice’s picture

Thanks @larowlan!

should we wrap this in a try catch and then do the switch back in a finally?

just in case something in validation throws an exception?

This is wise. I added a finally block but left off the catch block for the reasons in the patch comment. This means that the account will be switched back whether there is an exception or not and the exception will bubble up just as it does today.

Every row import() is wrapped in a try/catch in MigrateExecutable, which captures the exception and stores it in the migration messages table. If I caught it, I would have had to either silence the exception or duplicate that logic.

heddn’s picture

Status: Needs review » Reviewed & tested by the community

All feedback addressed. Back to RTBC.

catch’s picture

This looks like probably the only fix available within migrate, but feels like it's an issue with filter format validation in general.

FilterFormat::access() accepts an $account parameter, however we can't pass that to ::validate().

This wouldn't have come up when we were doing validation in form API, it's because we're also trying to handle REST requests, but migrate is neither and feels like this kind of use case has been missed.

catch’s picture

Status: Reviewed & tested by the community » Needs review

There's another problem here as well. There's no guarantee that the entity owner has access to the text format at the point the content is migrated, for various reasons:

1. The target site has different permissions for the text format.

2. The user is no longer in the role that has permissions for the text format.

3. The content was edited by an admin user, and the text format was changed to one that the entity owner doesn't have access too.

... and etc.

So I think this means we'd need a way for validation to bypass the access check in text format validation altogether. Moving back to needs review for more discussion.

gabesullice’s picture

This wouldn't have come up when we were doing validation in form API, it's because we're also trying to handle REST requests, but migrate is neither and feels like this kind of use case has been missed.

Migrate feels like an exceptional case. Validating entities instead of specific forms means Drupal can safely handle updates by users via HTTP requests (which includes form POSTs), that seems like right use case to optimize for. Everything else I can think of falls into developer-only scenarios. E.g. drush hooks, cron jobs, migrations, etc.

This is a best-effort resolution which makes a reasonable assumption IMO. Entity validation during migrations is an opt-in feature too, which makes it even more acceptable. Just disable it if its an enormous problem.

1 & 2: most Drupal-to-drupal migrate stuff in core assumes you're upgrading from D7 to D8+ without making any changes so this is not out of the ordinary. Whether that was a good idea is a much, much bigger fish to fry.

3: This is likely a one-off situation. The DX will be:

  • see the validation error in the migrate messages
  • decide whether it's too much work to resolve
  • if it is
    • turn off validation for that migration
  • it it isn't, either
    • go into the source site and change the format/owner
    • or write a process plugin
heddn’s picture

The validation errors surfacing from 82.1-3 are exactly the types of errors I would want to surface. It gives me the chance to fix the data in D6/D7 before I migrate. Or fix the destination system if their permission model is incorrectly setup. Without the validation, the data is imported silently and it isn't until a few days/weeks later that an editor goes into an old page and finds they can't edit the data.

Also, while typically the issue is for filter format, it could also surface w/ field permissions as well. What if there is a field that only editable by the node owner? But not by other users?

And yes, turning on entity validation is optional. And turned off by default. Because of just these types of reasons we've been discussing.

m.stenta’s picture

Status: Needs review » Reviewed & tested by the community

Setting this back to RTBC. I believe @catch's concerns were addressed by @gabesullice's and @heddn's comments.

marvil07’s picture

Small +1 on the re-roll, I arrived to exactly the same patch, and it looks good, even if I have not examined it in details, so it testbot confirms it is likely ready.

  • catch committed 8065ee3 on 9.3.x
    Issue #3134470 by heddn, gabesullice, sudiptadas19, ridhimaabrol24,...
catch’s picture

Status: Reviewed & tested by the community » Fixed

#82 and #83 makes sense - either don't validate, or validate with the correct user.

Committed 8065ee3 and pushed to 9.3.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

quietone’s picture

Publish CR