Follow-up to #2225775: Migrate Drupal 6 core node translation to Drupal 8 and #2669964: Migrate Drupal 7 core node translations to Drupal 8.

Problem/Motivation

In Drupal 6 and Drupal 7, translations of nodes have different node IDs. In Drupal 8, they have the same IDs. When we migrate node data from Drupal 6 or Drupal 7 to Drupal 8, a lot of data may point to things that no longer exist.

Eg: In Drupal 6, suppose node/3 is in English, and node/5 is a translation of it into French. Once migrated, there will be only a node/3 in Drupal 8. What happens to:

Proposed resolution

Some ideas:

  • Add a migrate process that looks up IDs in both the node and node_translation, so references work. We should try to do this in a way that doesn't break different kinds of translations, eg: Drupal 8 -> Drupal 8.
  • Add a new migration for example to add aliases for Drupal 6 translated nodes , so that links to node/5 work
  • Identify other related data that's involved.

Remaining tasks

Verify what happens in the above cases, and open sub-issues for them if not fixed (document here if they're supported, pointing to the test coverage).

Comments

vasi created an issue. See original summary.

vasi’s picture

Issue summary: View changes
penyaskito’s picture

Cleaned up tags.

I don't think we'll find a way for automatically solving this, but will leave this open for discussion.

Nothing could prevent users for having a different node/4, or having node/4 be a view, so automatically generating those aliases seems like a no-go to me.

vasi’s picture

Hmm...I think it would be nice to fix it by default. If someone wants to delete the alias and put a view there, they can still do that afterwards.

catch’s picture

Category: Task » Plan

Moving this to a plan,we'll need sub-issues for the actual work:

- fixing entity references to point to the correct node feels pretty critical
- it should be relatively predictable to look for path aliases pointing to translations, then migrate them to point to the single entity instead.
- not sure about menu links, there are different ways to set up multilingual menus, and we might end up doing things like duplicating links in the same menu for every translation.
- Adding path aliases for un-aliased translation paths seems a bit trickier, also they should be redirects and we don't have those in core.

mikeryan’s picture

Status: Active » Postponed
mikeryan’s picture

Status: Postponed » Active
Gábor Hojtsy’s picture

This is definitely the next wave of big problem space following #2225775: Migrate Drupal 6 core node translation to Drupal 8 :) I think fixing some crucial things like entity references and path aliases would be important. Not sure all the use cases are possible but if we start with path aliases, let's say (which I think sound most approachable) we can get some real life feedback on that.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Gábor Hojtsy’s picture

Title: Handle data related to D6 translations with different IDs » [META] Handle data related to D6 translations with different IDs

Meta-ifying this given @maxocub opened #2827644: Fix path alias migration of translated nodes [D6] as a first step.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Gábor Hojtsy’s picture

Title: [META] Handle data related to D6 translations with different IDs » [META] Handle data related to Drupal 6 and 7 node translations with different IDs
Issue summary: View changes

Updated to apply to Drupal 7 too in light of #2669964-100: Migrate Drupal 7 core node translations to Drupal 8.

catch’s picture

Priority: Major » Critical
Issue tags: +Migrate critical

Bits of this are proper data integrity issues. They won't affect mono-lingual sites but they could easily escape detection on multi-lingual migrations since reference fields are permissive about references being stale.

Gábor Hojtsy’s picture

maxocub’s picture

Gábor Hojtsy’s picture

Catch proposed #2850085: Redirects for translation set migration path in Drupal 6 and 7 to do redirects too for the sake of menu items, in-content links, etc.

catch’s picture

Issue tags: +Triaged D8 critical

Discussed with @alexpott, @cilefen, @cottser, @laurii, and @xjm and we agreed on the critical status since there's no way to recover the node IDs within a migrated 8.x site to fix any content issues you might have.

We probably need to break this issue down to individual sub-issues and figure out the priorities of each though.

heddn’s picture

Status: Active » Fixed

Moving the critical down to the child issue that is the final task. And closing this.

Gábor Hojtsy’s picture

Status: Fixed » Active

Well, the issue summary mentions entity references for example (as well as menu items, node queues, user ratings, etc) which did not even have a subissue yet. I think some core solutions may be in order as well as some infrastructure support for contrib modules like node queue, user ratings, etc.

catch’s picture

Issue summary: View changes
mikeryan’s picture

OK, we need to go through the list in the IS and review each item to see if it's covered already - if not, open a subissue:

  • Entity references that point to node/5?
  • Links in text fields that point to node/5?
  • Menu items that reference node/5?
  • User ratings that point to node/5?
  • Node queues listing node/5?
  • Node access data on node/5?

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

heddn’s picture

Issue tags: +Migrate Drupal

Inventing a tag to mark this as pertaining to migrate_drupal and therefore blocking its stable release.

heddn’s picture

heddn’s picture

Who is able to do the review requested in #21? That review could lead to a whole can of worms and this is blocking stability of migrate_drupal.

heddn’s picture

Discussed in weekly migrate meeting. We agree this meta is migrate critical. We'll try add sub-issues with the idea that as long as none of them are API / BC breaks, we might not need to mark them all as migrate critical.

Notes from the discussion:
Entity references that point to node/5 - this should be solved in core
Links in text fields that point to node/5 - possibly solved in contrib (I'm thinking pathologic)
Menu items that reference node/5 - this should be solved in core
User ratings that point to node/5? - possibly solved in contrib
Node queues listing node/5? - possibly solved in contrib
Node access data on node/5? - Probably fixed in core, but I'm not sure it is possible or if the node access system has changed so much between d6/d7 and d8 that there would be much benefit.

Gábor Hojtsy’s picture

@heddn: if there is node access data migrated, we should migrate it to the right place; D8 allows node access per language also. Keep in mind as well that the list in the issue summary is a list of examples and not conclusive.

Gábor Hojtsy’s picture

I think the first step here is to have some data store for mappings that can be permanent for cases that cannot be handled within the migration process. #2850085: Redirects for translation set migration path in Drupal 6 and 7 is a use case for that and it aims to provide a solution that others can use AFAIS.

The other part of the problem is for things that can be solved within the migration process. I imagine migrate would have a way to react to ID merges and run followup migrations for other data. I am far from well versed enough in migrate to propose how this could work. I also don't think it is worth opening followup issues for individual data changes if the underlying mechanism is not there yet. We need an underlying mechanism first. Please correct me if I am missing something here.

catch’s picture

Issue summary: View changes
phenaproxima’s picture

Okay, here's what I came up with in Vienna as a plan of attack:

  • Entity references that point to node/5?

    Let's create more migrations to deal with this. They would have to run after the node migrations (d6_node:*, d7_node:*, and their ilk), which we can accomplish using the existing migration dependency system. They could easily loop through all entity reference fields, and then simply use the migration_lookup plugin to adjust the value.

  • Links in text fields that point to node/5?

    This is covered by #2850085: Redirects for translation set migration path in Drupal 6 and 7.

  • Menu items that reference node/5?

    Since menu items, as far as I know, store their canonical path, we can take a similar approach as with entity reference fields.

  • User ratings that point to node/5?

    If it originates in contrib space, contrib space will need to provide migrations. They can use the entity reference and menu item migrations as examples/templates.

  • Node queues listing node/5?

    Ditto.

catch’s picture

#30 sounds very promising!

maxocub’s picture

Issue tags: +Vienna2017

Tagging and looking at creating sub-issues for #30.

Gábor Hojtsy’s picture

Agreed with @catch, #30 looks fine, good plan!

quietone’s picture

Made child issues for

entity_refenences: #2912348: Handle entity_references related to Drupal 6 and 7 node translations with different IDs
menu_items: #2912353: Handle menu_items related to Drupal 6 and 7 node translations with different IDs

Links already has an issue and user_ratings and node queue are in contrib, so do not require an issue here.

quietone’s picture

Issue summary: View changes

Just adding the issue numbers to the IS, seems an easy way to see where we are at with this.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

maxocub’s picture

maxocub’s picture

Priority: Critical » Major
Issue tags: -Migrate critical

I've read through this issue to see if we could close it.

The only thing I found that has not yet been addressed is the node_access, but I can't find any node_access migrations and I don't know if this is intentional.

Also, we have to remember that the list in the IS was not exhaustive and that there could be other migrations needed.

For those two reasons, I don't feel confortable closing this meta. But one thing we could do is to keep it open, lower the priority and keep looking for other places where migrations for handling different IDs are needed. Hopefully we won't find any.

catch’s picture

Status: Active » Reviewed & tested by the community

node_access doesn't need a migration because it can be rebuilt from scratch.

I also think we might be done here, moving to RTBC for a bit more visibility. If no-one thinks of anything in a couple of weeks, I think we could mark this fixed.

catch’s picture

Status: Reviewed & tested by the community » Fixed

That's a week and a bit with no comments, marking fixed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.