Problem/Motivation
In Drupal 7, the entityreference module allowed to create multiple field instances of the same field. The migration here "works" but then any time an entity without any data attempts to set an entity reference value an InvalidArgumentException is thrown in ContentEntityBase.
InvalidArgumentException: Field <field name> is unknown.
A comparison between getting field definitions directly from entity_field.manager reveals that the field does in fact exist on the bundle, but getting the field definition on the entity itself fails. I found that if I reset the field definitions before the exception, then it tries to go further, but runs into a similar exception in typed_data_manager.
Furthermore this is only an issue for one of the bundles. The other bundle works fine.
Steps to reproduce (as detailed in a test-only patch):
- Add field_reference_3 field_config
- Add field_reference_3 field_config_instance to both article and forum node bundles.
- Run the appropriate migrations.
- Load node 2 (an article node) e.g.
$node = Node::load(2); - Attempt to set an initial reference e.g.
$node->set('field_reference_3', [5]);or$node->get('field_reference_3')->appendItem()->set('field_reference_3', [5]);or$node->field_reference_3 = ['target_id' => 5]; - An exception is thrown and we can see that the field definition doesn't exist on the node, but does exist on field manager.
This might be why #2814963: Support Drupal 7 entity reference fields is still open/postponed even though plugins exist for it?
Proposed resolution
Reset the entity cache of the entity type that a field instance is attached to after the field instance is migrated.
Remaining tasks
1. Review proposed resolution
2. Reword test comments to accurately describe the issue and problem
User interface changes
Unknown
API changes
Unknown
Data model changes
Unknown
| Comment | File | Size | Author |
|---|---|---|---|
| #53 | interdiff-3008202-49-53.txt | 794 bytes | voleger |
| #53 | 3008202-53-test-only.patch | 11.41 KB | voleger |
| #44 | 3008202-44.patch | 14.7 KB | durgeshs |
| #44 | interdiff_42-44.txt | 2.85 KB | durgeshs |
| #42 | interdiff_39-42.txt | 1.03 KB | durgeshs |
Issue fork drupal-3008202
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
mradcliffeUploaded a slightly better patch.
Comment #3
mradcliffeI'm more confident this is an issue with migration rather than Entity API.
I tried to reproduce directly with the Entity API by creating 3 content types, 2 entity reference fields of the same attached to 2 of those node types, creating 1 node for each of the content types, saving, then setting the 2 nodes with reference fields. I didn't run into an issue with setting values.
Not really any value in uploading patch with that test, but here's the gist based on a FieldKernelTest:
Comment #5
mradcliffeComing back to this issue.
- I fixed the failing test that happened when I added a new field config/instance.
- Rebased off of 8.8.x and fixed the config / test changes that landed recently in 8.7.x. This makes the interdiff kind of useless as it had problems dealing with the changes to core.
This should only have the one isolated test failure.
I think that narrowing things down, maybe add field data for forum (or article), and add an assert about that field definition to see if it only happens with reference fields migrated without data?
Comment #6
mpp commentedComment #7
mpp commentedTest only so setting back to needs work.
Comment #8
mradcliffeAnother number to increment.
Also, I'm manually adding the test after I post a comment rather than flipping status.
Comment #9
mradcliffeI'm fairly certain that this is the wrong solution, but it does fix the issue. I write this because of the research I did earlier about trying to reproduce in the entity system, in which I could not. However this could serve as a core patch for someone that needs migrations that don't lead to a broken site.
This probably has some performance impact.
Comment #11
mradcliffeResetting the storage cache seems to help, but I'm not sure how that could be done system-wide for all entity reference fields on migration.
Also, the other issue with the approach I had, is that there are non-properties on nodes and users such as "user_cancel_notify" or "original", which mean this probably gets called often.
Comment #13
mradcliffeRe-rolling #8 and adjusting field_config, field_storage_config counts in Upgrade7Test. It should be correct, but the merge conflict was a bit tricky. I removed the "fix" that I added in #9.
I think that saving the entity should work during migration to address this, but want a clean test-only patch to work with.
Comment #14
mradcliffeNope, I screwed up that merge. Sigh. Ended up in vim because phpstorm ground to a halt trying to syntax check.
Comment #15
mradcliffeAnd of course I screwed THAT patch up by including the bad fix. Sigh. I'm tired and it's late.
Comment #16
mradcliffeI still must have messed the merge up because all the migrate tests are now failing. :(
Comment #17
mradcliffeI think this should get the patch back to the state it was in #8.
Comment #18
mradcliffeTest only patch needs another re-roll.
2 new fields and field instances got added to drupal7.php. The interdiff failed.
Comment #19
mradcliffeResetting the entity storage cache(s) of the entity type of a field instance after importing the field instance should do the trick here.
This could potentially cause a site to be slower post-migration if the migration was run in multiple parts as each entity will need to refetch the field definitions as needed. But that data was stale and bad anyway.
Comment #20
mradcliffeUpdated out-of-date issue summary.
Comment #21
mradcliffeI think that's why I had "refactor test" in the issue summary ;-)
Adding that back as a remaining task in the issue summary, but reworded to better clarify the intent of why I originally had it as a remaining task.
Comment #24
mradcliffeUploading a 8.8.x fix only patch so that I can apply it in composer.json.
Comment #25
dhirendra.mishra commentedPls make sure to move status to needs review after patch
Comment #26
mradcliffeThis still needs to be Needs work because the relevant patch is #19. I probably should have re-uploaded it in #24 as well though. The patch in #24 is not for testing or review, but mainly for my own use with a stable version of Drupal.
Current status is to refactor the test assertions so that it's not a proof-of-concept test and to re-roll based on changes to core.
Comment #28
quietone commentedReroll and update of the patch in #19. The patch in #19 added a new test, which has been removed and the assertions moved to the relevant existing tests, MigrateField, MigrateFieldInstance and MigrateNode.
Comment #30
durgeshs commentedComment #31
durgeshs commentedPrevious patch no longer to applied

So I have rerolled previous patch. Kindly check it.
Comment #32
durgeshs commentedWaiting for test cases. I am working for test cases.
Comment #33
quietone commented@durgeshs, thanks for the reroll. There is no need to output the results of a git pull and git apply, we can see from the test run that the patch does not apply. To aid the reviewer please provide at least an interdiff or if that fails a diff, See Creating an interdiff in the handbook.
Comment #35
durgeshs commented@quietone,
I have added interdiff and new patch with resolve test case under core/modules/field/tests/src/Kernel/Migrate/d7.
Comment #37
quietone commentedFound two out of scope changes that were introduced by me earlier on.
out of scope
Out of scope
Good. I checked the field definition and it does have a cardinality of 1.
Why has the error count increased? Is is supposed to? What is the error message?
Comment #38
durgeshs commentedComment #39
durgeshs commented@quietone,
I have done changes as per #37, kindly review it.
Comment #40
quietone commented@durgeshs, thanks for fixing the out of scope changes I introduced.
This will allow the test to pass but I don't think the test itself is correct. This issue is to migrate more that 1 entity reference field so getting a new errors seems wrong. The changes here that allow the test to pass need to be explained.
Settings to NW for #37.4 and the above change to the test.
Comment #41
durgeshs commented@quietone, Thanks for feedback.
Sure, I will check and resolve it.
Comment #42
durgeshs commented@quietone,
I have checked code and found that there are getting one extra message when migrate more than 1 entity reference field. This message occurring because of the following code
import(Row $row, array $destination_id_values),$destination_id_values is getting blank array for first entity reference filed.
Please feel free to let me know if there are any best solution.
Meanwhile in the current patch, I have added a proper comment for this.
Comment #43
quietone commented@durgeshs, Yes, that is not unexpected, it will be failing somewhere in the import process. Just have to find out where. Maybe I can do some debugging on this in the next couple of days.
The comment before the tests on the error messages states that it is for text field instances not entity reference field. So, these changes need to be removed. Sorry about that.
The fixture add two instances of the field_reference_2 and we are only checking that one on the article is migrated. A similar test needs to be added for the forum bundle. Please add it a few lines down where the other 'forum' bundle field instances are tested.
Comment #44
durgeshs commented@quietone, Thanks for the feedback, I have addressed those two comments of #43 and will be waiting for your debugging input.
Comment #45
quietone commentedI manually tested today and wasn't able to produce an error. I followed the steps 1, 2 and 3 in the ISS then I used the UI to edit node 2 and add a reference. There were no errors in the log.
@mradcliffe, is this still a problem?
Comment #47
quietone commentedIs this issue still a problem?
There has been no activity here for 7 months.
Since we need more information to move forward with this issue, I am keeping the status at Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.
Comment #49
benjifisherBased on #45, it looks as though this issue was solved by some other changes in Drupal.
I tried to confirm this. I applied the test-only patch from #18 to commit 45b25f08b7 (2019-12-13). I installed PHP 7.2 and did
composer install(with Composer 1.10.13) to get PHPUnit 6.5.14. After this experiment in time travel, I got the same test failure reported by the testbot in #18: "Failed asserting that an array has the key 'field_reference_3'."Then, with some difficulty, I rerolled the test-only patch so that it applies to 9.3.x. With PHP 7.3 and PHPUnit 8.5.15, I get the same failure.
Part of the reroll is removing the reference to
NodeCommentCombinationTrait, which was removed in #3111344: Remove deprecated code from migrate module. The test-only patch did not use the method defined in that trait, so no related changes were needed.There are a lot of things I have not done: I have not looked at the test to decide what it does. Maybe it is bogus. Maybe it is not testing what the issue summary says. I have not compared the failing test to the tests added by @quietone in #28 and later patches. I have not done any manual testing.
I did check that the patch in #18 is a test-only version of the patch in #19.
I am removing an unhelpful screenshot from the issue summary.
I think we have a failing test, so I am setting the status back to NW.
P.S. It is also quite possible that I made a mistake while rerolling the patch. But then it would be quite a coincidence that I get the same test failure.
Comment #53
volegerRerolled #43 test-only patch
Comment #54
volegerComment #57
joelpittet