Problem/Motivation

When migrating entity reference fields from a multilingual D7 site to a multilingual D8 site, I get the following error:

"Value is not a valid entity." line 106 on core/lib/Drupal/Core/Entity/Plugin/DataType/EntityReference.php

This happens because core expects the entity reference value expects either a scalar value(like an ID) or an object typed as EntityInterface and the migration is sending it an array with ID and langcode.

Proposed resolution

I imagine one solution would be to allow sending both ID and langcode as parameters for an entity load, but I'm not sure how that would work exactly. I have a workaround for now that I'll describe in detail on the comments bellow.

Comments

jmoreira created an issue. See original summary.

jmoreira’s picture

Task: Migrate multilingual content types A and B from D7 to D8. Content type A has a reference to content type B.

Note: In this scenario, the Content Translation module is being used for D7, so each translation is an individual node.

My current solution/workarounds for this issue:

  • Create 4 migrations: original content type A, translations of content type A and original content type B, translations of content type B. I needed to do that because if I specify that I want to migrate translations, the original node(n.nid = n.tnid) isn't migrated.
  • Migration for original content type A references original content type B with a migration process plugin.
  • Migration for translations of content type A references translations of content type B with a migration process plugin.
  • Need patch to remove langcode as an ID.

If there's a better solution/workaround for this, I would be glad to know and document it somewhere. Here are a sample of my migration config files:

File: migrate_plus.migration.node_a.yml

id: node_a
label: Node A
migration_group: example
source:
  plugin: d7_node
  key: migrate
  node_type: node_a
destination:
  plugin: entity:node
process:
  title: title
  langcode: language
  type:
    plugin: default_value
    default_value: node_a
  field_reference_node_b:
    plugin: iterator
    source: field_reference_node_b
    process:
      target_id:
        plugin: migration
        migration: node_b
        source: target_id

File: migrate_plus.migration.node_a_translations.yml

id: node_a_translations
label: Node A Translations
migration_group: example
source:
  plugin: d7_node
  key: migrate
  node_type: node_a
  translations: true
destination:
  plugin: entity:node
  translations: true
  content_translation_update_definitions:
    - node
migration_dependencies:
  required:
    - node_a
process:
  nid:
    plugin: migration
    migration: node_a
    source: tnid
  title: title
  langcode: language
  content_translation_source: source_langcode
  type:
    plugin: default_value
    default_value: node_a
  field_reference_node_b:
    plugin: iterator
    source: field_reference_node_b
    process:
      target_id:
        plugin: migration
        migration: node_b_translations
        source: target_id

File: migrate_plus.migration.node_b.yml

id: node_b
label: Node B
migration_group: example
source:
  plugin: d7_node
  key: migrate
  node_type: node_b
destination:
  plugin: entity:node
process:
  title: title
  type:
    plugin: default_value
    default_value: node_b

File: migrate_plus.migration.node_b_translations.yml

id: node_b_translations
label: Node B translations
migration_group: example
source:
  plugin: d7_node
  key: migrate
  node_type: node_b
  translations: true
destination:
  plugin: entity:node
  translations: true
process:
  nid:
    plugin: migration
    migration: node_a
    source: tnid
  title: title
  langcode: language
  type:
    plugin: default_value
    default_value: node_b
gábor hojtsy’s picture

I don't see how comment #2 is related to the original problem where you say entity references should accept other kinds of values.

I needed to do that because if I specify that I want to migrate translations, the original node(n.nid = n.tnid) isn't migrated. Is there a better way to do this?

Yes, so d7_node_translation does not specify the node migration as a dependency (for some reason), but if you look at how its tested (MigrateNodeTest), the node migration is ran before the translation migration. The node migration will migrate the original language.

jmoreira’s picture

@Gabor, comment #2 was to show the solution/workaround I got so far. For the "main content type" I do the migration using the tnid as the new nid and it works fine. The problem happens when I add the node reference field, because out of the box when I migrate the referenced content and use the migration process to get the referenced nid, the target_id is returned as an array of id and langcode, instead of an int/string with the id.

So, the workaround outlined in #2 was to remove langcode as an ID(with the patch attached) and import each referenced entity with a new nid. Because I thought(wrongly) I would have problems using repeated nid's as target ID.

I did some new tests and it actually works if I migrate all referenced translations to one node entity, so that isn't an issue, I updated the comment my latest solution, sorry about the mix up.

The actual issue now is the error in the description. The patch on #2 makes it work on my case, but I imagine it's a bad idea to just drop the langcode as an destination ID? Maybe we should do this conditionally? Or at the getValue function for the entity reference field use only the first value if an array is given?

gábor hojtsy’s picture

I think this may be too much in the weeds of migrate for me to practically help :/ Know multilingual much better than migrate. But if the langcode is dropped, how is the correct translation identified?

jmoreira’s picture

The language is dropped as a destination id but it's still used to create/update the entity. So for an example in D7 where nid 123 is the original en content and 124 is the fr translation, the mapping table would look something like:

   souceid1 | destinationid1
    123     |        1
    124     |        1

And then the entity destination plugin uses the given langcode to decide if it will create/update a node or a translation. I imagine that in this case where in the D7 source translations have different nid's we don't need the langcode as a destinationid2, but maybe it would be useful in a setup where the D7 source has the same nid for all translations, like with Entity Translation, I think.

gábor hojtsy’s picture

Yeah I would generally say removing meta information that distinguishes an element is not a good idea. Making systems work with that meta information is better :)

jmoreira’s picture

That makes sense, though it looks like that the Entity Reference field doesn't need the language to select the proper translation for the given field, so I'm not sure why we need to send the language as an ID to the destination. Maybe we could just have it as an ID for mapping?(Not sure if such concept exists in the Migrate API)

samirjusic’s picture

I am having the exact issue of not being able to import EntityReference fields (with multiple values) that have translations. The default language migration works fine, but running the translation migration either fails OR it produces these new bizarre referenced items with really long titles for every entity it encounters but can't find (I suppose).

Applying the patch outlined in comment #2 does fix the import of translated content items.

I am curious if someone can point out what is the intended migration config that should work without the patch? I do see that the target ID from the sub-migration has both the ID and a language code. So are we supposed to split it with another plugin and then use it to point to the right destination?

samirjusic’s picture

One note, though: It seems that the patch works if its run on *only* the migration scripts that deal with the translation.

The first run of the migration (that goes through the default language migration) is to be run with the existing code untouched (I'm on 8.3.2). Then the patch is to be applied and the upgrade_d7_node_translation_NODE_TYPE.yml is run with that. This way I was able to complete the migration for the migrated content.

However, if I apply the patch and run the first migration as well with it (upgrade_d7_node_NODE_TYPE.yml) then it seems to me that the system generates these weird nodes that it would generate only in the 2nd step when running the _translate_NODE_TYPE.yml (basically the errors I was experiencing when trying to complete the migration for the translated content would happen right away with the first run of the migration script)

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

nightlife2008’s picture

This patch breaks the path migration's node translation lookup process plugin.

It needs the langcode for a translation migration (destid2) to set the correct language for the path alias.

nightlife2008’s picture

As a follow-up on my previous comment, I've retested my own bugfix patch and decided to share it with you guys.

I've been using the various plugins and the only one returning a "faulty" value for an EntityReference is the MigrationLookup plugin.

However, the current patch breaks the UrlAlias translation lookup process plugin, because there isn't a langcode column anymore to depend on, to create the D8 Url alias entity.

In my opinion the patch is as easy as changing 1 character, and to "reset" the destination_ids array, even if there are more then 1 items in the array. This returns a single ID everytime, and thus not removes the metadata (langcode) from the DB scheme, but only inside the method that uses the Id lookup.

Feel free to comment on my fix.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

quietone’s picture

Issue tags: +migrate-d7-d8

tagging

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

Status: Active » Closed (outdated)
Issue tags: +Bug Smash Initiative

@jmoreira, thanks for filing this issue and creating a patch.

A migration for entity references for translated nodes was created in #2912348: Handle entity_references related to Drupal 6 and 7 node translations with different IDs and committed on Apr 19 2018. That will will have fixed this problem. There hasn't been any activity on this since before that commit which also leads me to believe this has been fixed.

I'm going to close this as outdated. If I am wrong, reopen this issue by setting the status to Active and explain what is not working, what error message you are getting and anything you think is relevant.

Thanks!

shiraz dindar’s picture

I realize this is an old issue, so I'm not going to reopen it, but for the benefit of others, I want to comment that I had the same issue as described by the O.P, and the patch #13 resolved it. So I would say the comment in #19 that this issue was fixed with that commit is incorrect. I'm fine with the patch. I just want to let anyone that finds this issue know.

majorrobot’s picture

Version: 8.9.x-dev » 10.2.x-dev
Status: Closed (outdated) » Needs review

If I am wrong, reopen this issue by setting the status to Active and explain what is not working, what error message you are getting and anything you think is relevant.

Ack, I hate to reopen this issue, but I think it is warranted. If I should not, let me know! I'm having the same experience as OP, as well as @Shiraz Dindar above.

I'm running Drupal 10.2.5 and I'm migrating from another site on 10.2.5. Though I'm using the migrate_drupal_d8 contrib module, it still calls the core Migration Lookup plugin. In my case, I have an image field migrating to an identical image field. Both sites have Content Translation enabled for the content type. It does not seem to matter, but I have the image field configured not to be translatable.

#2912348: Handle entity_references related to Drupal 6 and 7 node translations with different IDs has some similarities, but I think it is actually a different issue.

I understand that issue to solve a problem where entity reference migrations aren't looking up the correct original target_ids. The issue here is that entity references end up with 2 destination ids: the target_id and the langcode. The target_id is correct. However, EntityReference->setValue() only accepts a single scalar value.

Thus, I'm still getting
"Value is not a valid entity." line 106 on core/lib/Drupal/Core/Entity/Plugin/DataType/EntityReference.php

The patch in #13 solves the issue for me, too. The solution in the patch seems a bit heavy-handed, but I don't know other use cases well enough to say whether or not the patch would break other migrations. It seems a bit dangerous, but I can't say for sure:

     if ($destination_ids) {
-      if (count($destination_ids) == 1) {
+      if (count($destination_ids) >= 1) {
         return reset($destination_ids);
       }
       else {
needs-review-queue-bot’s picture

Status: Needs review » Needs work

The Needs Review Queue Bot tested this issue.

While you are making the above changes, we recommend that you convert this patch to a merge request. Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)

quietone’s picture

Version: 10.2.x-dev » 11.x-dev
Issue tags: +Needs tests

This needs to be fixed on 11.x (our main development branch) first, and are then back ported as needed according to our policies. Also, 10.2 is in security mode now.

Thanks for the extra information in #21. That seems useful. We do need a test here to demonstrate the problem.

quietone’s picture

Status: Needs work » Postponed

The Migrate Drupal Module was approved for removal in #3371229: [Policy] Migrate Drupal and Migrate Drupal UI after Drupal 7 EOL.

This is Postponed. The status is set according to two policies. The Remove a core extension and move it to a contributed project and the Extensions approved for removal policies.

The deprecation work is in #3522602: [meta] Tasks to remove Migrate Drupal module and the removal work in #3522602: [meta] Tasks to remove Migrate Drupal module.

Migrate Drupal will not be moved to a contributed project. It will be removed from core after the Drupal 12.x branch is open.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

quietone’s picture

Status: Postponed » Closed (won't fix)

The Migrate Drupal Module and Migrate Drupal UI are deprecated and they are not in Drupal 12.0.0.

Issues for these modules should now be on the 11.x branch. And the changes are limited to critical and major bug fixes. Other changes are allowed at the discretion of the core Release Managers in consultation with the Migrate subsystem maintainers.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.