Follow-up to #2225775: Migrate Drupal 6 core node translation to Drupal 8 and #2669964: Migrate Drupal 7 core node translations to Drupal 8.
Problem/Motivation
In Drupal 6 and Drupal 7, translations of nodes have different node IDs. In Drupal 8, they have the same IDs. When we migrate node data from Drupal 6 or Drupal 7 to Drupal 8, a lot of data may point to things that no longer exist.
Eg: In Drupal 6, suppose node/3 is in English, and node/5 is a translation of it into French. Once migrated, there will be only a node/3 in Drupal 8. What happens to:
- Entity references that point to node/5? #2912348: Handle entity_references related to Drupal 6 and 7 node translations with different IDs
- Links in text fields that point to node/5? #2850085: Redirects for translation set migration path in Drupal 6 and 7
- Menu items that reference node/5? #2912353: Handle menu_items related to Drupal 6 and 7 node translations with different IDs
- Node counter for node/5? #2930101: i18n / statistics - node counter not updated for translations
- User ratings that point to node/5? - In contrib, nothing to do here.
- Node queues listing node/5? - In contrib, nothing to do here.
- etc. (endless list with contributed modules)
Proposed resolution
Some ideas:
- Add a migrate process that looks up IDs in both the node and node_translation, so references work. We should try to do this in a way that doesn't break different kinds of translations, eg: Drupal 8 -> Drupal 8.
- Add a new migration for example to add aliases for Drupal 6 translated nodes , so that links to node/5 work
- Identify other related data that's involved.
Remaining tasks
Verify what happens in the above cases, and open sub-issues for them if not fixed (document here if they're supported, pointing to the test coverage).
Comments
Comment #2
vasi CreditAttribution: vasi at Evolving Web commentedComment #3
penyaskitoCleaned up tags.
I don't think we'll find a way for automatically solving this, but will leave this open for discussion.
Nothing could prevent users for having a different node/4, or having node/4 be a view, so automatically generating those aliases seems like a no-go to me.
Comment #4
vasi CreditAttribution: vasi at Evolving Web commentedHmm...I think it would be nice to fix it by default. If someone wants to delete the alias and put a view there, they can still do that afterwards.
Comment #5
catchMoving this to a plan,we'll need sub-issues for the actual work:
- fixing entity references to point to the correct node feels pretty critical
- it should be relatively predictable to look for path aliases pointing to translations, then migrate them to point to the single entity instead.
- not sure about menu links, there are different ways to set up multilingual menus, and we might end up doing things like duplicating links in the same menu for every translation.
- Adding path aliases for un-aliased translation paths seems a bit trickier, also they should be redirects and we don't have those in core.
Comment #6
mikeryanPostponed on #2225775: Migrate Drupal 6 core node translation to Drupal 8.
Comment #7
mikeryanComment #8
Gábor HojtsyThis is definitely the next wave of big problem space following #2225775: Migrate Drupal 6 core node translation to Drupal 8 :) I think fixing some crucial things like entity references and path aliases would be important. Not sure all the use cases are possible but if we start with path aliases, let's say (which I think sound most approachable) we can get some real life feedback on that.
Comment #10
Gábor HojtsyMeta-ifying this given @maxocub opened #2827644: Fix path alias migration of translated nodes [D6] as a first step.
Comment #12
Gábor HojtsyUpdated to apply to Drupal 7 too in light of #2669964-100: Migrate Drupal 7 core node translations to Drupal 8.
Comment #13
catchBits of this are proper data integrity issues. They won't affect mono-lingual sites but they could easily escape detection on multi-lingual migrations since reference fields are permissive about references being stale.
Comment #14
Gábor HojtsyComment #15
maxocub CreditAttribution: maxocub commentedI opened #2850984: Fix path alias migration of translated nodes [D7] with a patch.
Comment #16
Gábor HojtsyCatch proposed #2850085: Redirects for translation set migration path in Drupal 6 and 7 to do redirects too for the sake of menu items, in-content links, etc.
Comment #17
catchDiscussed with @alexpott, @cilefen, @cottser, @laurii, and @xjm and we agreed on the critical status since there's no way to recover the node IDs within a migrated 8.x site to fix any content issues you might have.
We probably need to break this issue down to individual sub-issues and figure out the priorities of each though.
Comment #18
heddnMoving the critical down to the child issue that is the final task. And closing this.
Comment #19
Gábor HojtsyWell, the issue summary mentions entity references for example (as well as menu items, node queues, user ratings, etc) which did not even have a subissue yet. I think some core solutions may be in order as well as some infrastructure support for contrib modules like node queue, user ratings, etc.
Comment #20
catchComment #21
mikeryanOK, we need to go through the list in the IS and review each item to see if it's covered already - if not, open a subissue:
Comment #23
heddnInventing a tag to mark this as pertaining to migrate_drupal and therefore blocking its stable release.
Comment #24
heddnBlocker for #2905736: Mark Migrate Drupal as stable
Comment #25
heddnWho is able to do the review requested in #21? That review could lead to a whole can of worms and this is blocking stability of migrate_drupal.
Comment #26
heddnDiscussed in weekly migrate meeting. We agree this meta is migrate critical. We'll try add sub-issues with the idea that as long as none of them are API / BC breaks, we might not need to mark them all as migrate critical.
Notes from the discussion:
Entity references that point to node/5 - this should be solved in core
Links in text fields that point to node/5 - possibly solved in contrib (I'm thinking pathologic)
Menu items that reference node/5 - this should be solved in core
User ratings that point to node/5? - possibly solved in contrib
Node queues listing node/5? - possibly solved in contrib
Node access data on node/5? - Probably fixed in core, but I'm not sure it is possible or if the node access system has changed so much between d6/d7 and d8 that there would be much benefit.
Comment #27
Gábor Hojtsy@heddn: if there is node access data migrated, we should migrate it to the right place; D8 allows node access per language also. Keep in mind as well that the list in the issue summary is a list of examples and not conclusive.
Comment #28
Gábor HojtsyI think the first step here is to have some data store for mappings that can be permanent for cases that cannot be handled within the migration process. #2850085: Redirects for translation set migration path in Drupal 6 and 7 is a use case for that and it aims to provide a solution that others can use AFAIS.
The other part of the problem is for things that can be solved within the migration process. I imagine migrate would have a way to react to ID merges and run followup migrations for other data. I am far from well versed enough in migrate to propose how this could work. I also don't think it is worth opening followup issues for individual data changes if the underlying mechanism is not there yet. We need an underlying mechanism first. Please correct me if I am missing something here.
Comment #29
catchComment #30
phenaproximaOkay, here's what I came up with in Vienna as a plan of attack:
Entity references that point to node/5?
Let's create more migrations to deal with this. They would have to run after the node migrations (d6_node:*, d7_node:*, and their ilk), which we can accomplish using the existing migration dependency system. They could easily loop through all entity reference fields, and then simply use the migration_lookup plugin to adjust the value.
Links in text fields that point to node/5?
This is covered by #2850085: Redirects for translation set migration path in Drupal 6 and 7.
Menu items that reference node/5?
Since menu items, as far as I know, store their canonical path, we can take a similar approach as with entity reference fields.
User ratings that point to node/5?
If it originates in contrib space, contrib space will need to provide migrations. They can use the entity reference and menu item migrations as examples/templates.
Node queues listing node/5?
Ditto.
Comment #31
catch#30 sounds very promising!
Comment #32
maxocub CreditAttribution: maxocub for Acquia commentedTagging and looking at creating sub-issues for #30.
Comment #33
Gábor HojtsyAgreed with @catch, #30 looks fine, good plan!
Comment #34
quietone CreditAttribution: quietone as a volunteer commentedMade child issues for
entity_refenences: #2912348: Handle entity_references related to Drupal 6 and 7 node translations with different IDs
menu_items: #2912353: Handle menu_items related to Drupal 6 and 7 node translations with different IDs
Links already has an issue and user_ratings and node queue are in contrib, so do not require an issue here.
Comment #35
quietone CreditAttribution: quietone as a volunteer commentedJust adding the issue numbers to the IS, seems an easy way to see where we are at with this.
Comment #37
maxocub CreditAttribution: maxocub for Acquia commentedAdded #2930101: i18n / statistics - node counter not updated for translations to the IS.
Comment #38
maxocub CreditAttribution: maxocub as a volunteer commentedI've read through this issue to see if we could close it.
The only thing I found that has not yet been addressed is the node_access, but I can't find any node_access migrations and I don't know if this is intentional.
Also, we have to remember that the list in the IS was not exhaustive and that there could be other migrations needed.
For those two reasons, I don't feel confortable closing this meta. But one thing we could do is to keep it open, lower the priority and keep looking for other places where migrations for handling different IDs are needed. Hopefully we won't find any.
Comment #39
catchnode_access doesn't need a migration because it can be rebuilt from scratch.
I also think we might be done here, moving to RTBC for a bit more visibility. If no-one thinks of anything in a couple of weeks, I think we could mark this fixed.
Comment #40
catchThat's a week and a bit with no comments, marking fixed.