D6 migration testing with i18n

Thanks for investigating!

So... in answer to your questions...
1. No modules on either side related to workflow/revisions. On the D8 side, when I ran the migration, there were no contrib modules at all, actually. I have since added only Pathalogic.

2. There was zero content on the destination d8 side before I did the migration. All I did was:
- Install with Standard profile.
- Uninstall a few unneeded Core modules, such as Search and History
- Delete the provided Article and Page content types that come with the Standard install profile

3. Everything seems to have been migrated successfully, except for most of the Article content items missing their body in the latest revision. Three of the 25 articles were OK, and the other 22 were missing bodies in the latest revisions; the older ones had their revisions.

4. I also took a look at the vid 951. It is a different content type, and an older revision of that node (not the most current one). Let's see. On the old site, nid 666 has three revisions: 950, 951, and 954. On the new site, ... it looks like that node was indeed not migrated, assuming that the nid would be the same. Oops. I did not have any error messages about that node being missed.

As another note, that node in d6 is the Spanish translation of an English node. Something weird may be happening due to that?

OK, this is morphing into two separate issues.

a) The articles with missing bodies in the latest revision. This content type is untranslated. In D6, they are content type "article" (which I created there), and that content type was created for me by the migration.

An example is nid 13, which in D6 has revisions with vid 45, 49, 78, and 79. All revisions have an empty language code field (which in D8 was migrated into "und"). The bodies of the first three revisions came through fine in the migration, but the body of the final revision is empty (there is an entry in the node_revision table in D8 for vid 79, and the other fields are fine, but node_revision__body does not have an entry for that revision).

Hm. I had thought that all but 3 of the articles were OK, but I actually think that none of them was OK in the migration.

b) The translated nodes that had error messages during revision import. These are a separate content type, which is the only one that is translated on my site. And it sounds like you know about the node revision problem with translations. In any case... let's see. Node 666 in D6 is a translation of node 664. And it looks like node 664 came through successfully with an English and a Spanish translation. The Spanish one has no revision history, but it seems to have the latest revision. So, that's OK.

c) Other nodes had no problems. For instance, I have another content type called "news_item" and one called "page". None of them had language codes or translations in D6, but it was only "article" nodes that had import problems.

Hm. Actually, I take it back. I thought that only my articles were affected. Actually, quite a bit of the content didn't come through with the body fields -- I just found some more that were not articles that also had blank bodies in the latest revision.

So, I'm not sure what to do here. My blocks, menus, views, etc. didn't come through very well either (or not at all in the case of the views, which is apparently a known problem)... I was hoping to at least get the content migrated, but it may actually save me time to do it manually with copy/paste, rather than migrating and then trying to figure out what worked and what was broken. :(

I thought we were not going to end-of-life Drupal 6 until we had a viable migration strategy -- this doesn't seem very viable.

Sorry. I'm not ranting at you Mike -- you've done a LOT to get it to this point! Just wondering about the overall strategy from the project maintainers... I know no one wanted to maintain Drupal 6 (myself included), but if there's no way to migrate it to 8 that actually works, even just the content, for a site like mine (relatively uncomplicated, just a few CCK fields)... I'm not feeling very good about this decision to end-of-life Drupal 6. It may be feasible on a relatively small site to use copy/paste to migrate, but... not great for anything else.

OK, I still have the D6 database available. I'll try a migration into a clean latest-8.x-dev site and see what happens. I eventually abandoned the effort of migrating my d6 site and just did it manually (copy/paste). Which was very sad.

OK, I did a test. TL;DR -- I will post a summary in the next comment.

Summary of my D6 site... not a very big site:

- 6 content types using CCK:
-- article: fields of type text, file. Not translated. 23 nodes. 84 revisions total. Has comments.
-- download: Fields of type text, file, integer. Not translated. 11 nodes. 88 revisions. Has comments.
-- download_external: Fields of type text, integer, link. Not translated. 6 nodes. 18 revisions.
-- hodgblog: Fields of type file (one is image), taxonomy. Translated using core Content Translation + CCK Translation. The taxonomy vocabulary for this one is also translated. Also has comments. 80 nodes (40 in English and 40 Spanish translations). 151 revisions.
-- news_item: Has taxonomy. Not translated. Also has comments. 74 nodes. 141 revisions.
-- page: No fields (except the native D6 body field). Not translated. 20 nodes. 113 revisions.
There are 388 comments total.

- Several custom blocks
- One custom menu, plus links in Primary Links
- 4 views with several displays (pretty standard stuff, like a blog teasers page, recent articles, taxonomy list for the blog)
- One user

Here's what I did to test migrating this site today:
- Downloaded the 8.4.x-dev tarball from project/drupal
- Installed with Standard install profile, without update manager, using the UI
- Uninstalled a few core modules I didn't need, for speed (like search, history, help, tour), from the UI
- Deleted the Page and Article content types, from the UI
- Installed the 4 multilingual modules, plus Migrate, Migrate Drupal, and Migrate UI. Note: Did not add Spanish language.
- Cleared the dblog

Now I am ready to migrate:
- When I installed the Migrate UI module, it pointed me to go to /upgrade, so I did that.
- Read through the instructions, double checked that the field modules I should need are installed (like Image, File)
- Clicked Continue, entered database and file directory information.
- [PROBLEM] It tells me that a LOT of stuff that I think should be included is missing from migrate, such as filefield, imagefield, link, number, and date. Most of these are in Core now. I don't have any date fields. I checked a few of the missing D6 projects (filefield, imagefield, link), and didn't see anything about any d8 modules I could enable that would provide migration tips. So, proceeded.

Lots of errors (in no particular order):

a) [PROBLEM] Files not existing. This is because the directory I entered ended in a slash. Oops. I was not warned, and the slash was not removed. That is really dumb. I'll file an issue about that, if there isn't one already.

b) [PROBLEM]

Source ID node,full,hodgblog,field_images: Failed to lookup field type array ( 0 => 'filefield', 1 => 'hidden', ) in the static map.

I have no idea what that means. Maybe it's related to the image files being missing.

c) [PROBLEM]

Notice: Undefined index: value in Drupal\field\Plugin\migrate\process\d6\FieldInstanceSettings->transform() (line 37 of /opt/www/d8migrate/core/modules/field/src/Plugin/migrate/process/d6/FieldInstanceSettings.php) 

I'll file an issue about that, if there isn't one already.

d) [PROBLEM]

Notice: Undefined index: d6_i18n_user_settings in Drupal\migrate_drupal_ui\Form\MigrateUpgradeForm->buildConfirmForm() (line 1046 of /opt/www/d8migrate/core/modules/migrate_drupal_ui/src/Form/MigrateUpgradeForm.php) 

I'll file an issue about that.

e) [PROBLEM]

 	Source ID 951: Unable to get entity

These seem to be while migrating URL aliases was active (according to the Migrate UI in progress). Perhaps I had some aliases linked to nodes I deleted? Unsure. The last alias in my url_alias table in the d6 database is ID 690, amd the last node is ID 704, so I don't know what these IDs are.

f) [PROBLEM]

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'poplarware_d6.profile_fields' doesn't exist: SELECT COUNT(*) AS expression FROM (SELECT 1 AS expression FROM @profile_fields pf) subquery; Array ( ) 

Then the next message is

 	Operation on User profile field instance configuration failed

I don't know why it was looking for profile_fields but I don't have that table. I'll file an issue.

g)

Notice: Undefined index: allowed_html in Drupal\filter\Plugin\migrate\process\FilterSettings->transform() (line 45 of /opt/www/d8migrate/core/modules/filter/src/Plugin/migrate/process/FilterSettings.php) 

This is due to a missing text filter module that I had in D6.

I wasn't really warned very well about this problem... but to fix it, or at least get around it, I had to go to Configuration / Text formats and save the Full HTML and Filtered HTML text formats, to remove the null filters that didn't exist. Without doing that, content is not shown if it has those text formats.

OK. So let's see what the result is, as far as content.

1. [PROBLEM] Taxonomy. It looks like the English terms came through, but the translations of the translated vocabulary did not get migrated. The d6 site was using the i18ntaxonomy module to translate terms.

[PROBLEM] -- Another problem: although the taxonomy terms are shown on the nodes that are tagged with the two taxonomies, when I go to the term pages, the corresponding nodes are not shown there, for one of the two taxonomies. Ironically, the translated one is working (showing the English nodes), but the untranslated one isn't.

2. Nodes:
- The untranslated content items came through fine, with revisions. The display order of all of them is very weird (comments are above body), and images/files are missing (see above).
- The translated content items came through mostly fine, but with no revision history (I think no revisions on translations is a known issue and/or by design current, so that is OK).
[PROBLEM] However, the Spanish translations have no taxonomy terms on them (English or Spanish). They definitely do in the D6 site.

3. Custom Blocks: seem to have come through. They're all dumped into the main area of the page. Oh well.

So, in summary: With 8.4.x, the only i18n migration problem seems to be the taxonomy:
- The taxonomy terms translated with the d6 contrib i18ntaxonomy module did not come through as translated.
- The nodes in Spanish that were tagged with these terms did not come through tagged at all (with the English or Spanish versions of the tags).
- The other vocabulary, which wasn't translated: nodes came through tagged, but they don't show up on the taxonomy page for those tags.

There were several other migrate problems that are separate issues. I'll research/file issues.

Meanwhile, updating summary here.

New issue title...

Summary of issues related to the latest test (comment #14):

a) Translations of taxonomy did not come through. Existing issues:
#2225781: Migrate D6 i18n taxonomy vocabularies
#2784371: Migrate D6 i18n taxonomy term translation (but not yet localized translations)

b) Translated nodes with taxonomy terms didn't get the taxonomy terms attached. Filed new issue:
#2859297: Migrate taxonomy term references for D6 Node translations (I do not think it is part of the above two issues.)

c) Translated node revisions not migrated. Existing issues:
#2746541: Migrate D6 and D7 node revision translations to D8

d) Taxonomy pages... To review, on the news content type, the taxonomy terms are on the nodes, but the taxonomy page doesn't show the nodes. I just investigated this. The problem is that the default taxonomy term view has a filter that restricts to the current page's language. So, if I go to the /term/12 page, the language is English. But aside from my hodgblog nodes, all of the migrated nodes came through with language code "und", not "en". Is this a bug, or by design? It makes for a somewhat broken site.

e) Various errors and messages seen during the migrate process. Issues:
(new issue) #2859304: Show field type migrations correctly in Migrate Drupal UI
(existing issue -- added comment) #2562845-20: Validate file path on Credential form
(existing issue) #2827175: Undefined index: none FieldInstanceSettings.php:37
(new issue) #2859310: Migrate UI: Got undefined index errors in dblog from MigrateUpgradeForm
(new issue) #2859315: SQL error from profile_fields when migrating d6 (or d7) to d8 without Profile module

I think that this error noted in #14:
Source ID 951: Unable to get entity
was probably related to the translation revisions or translation taxonomy terms, because the only ID 951 I can find in my d6 database is a revision ID for one of the translated nodes. So ... not making a new issue for that error. Hopefully it's one of the above.

So aside from #14 (d) I think we can close this issue. Any thoughts on whether that should be a new issue, an existing issue, or it's not an issue even though it seems wrong to me?

I decided to file #2859359: Migrating content with missing language is problematic for the remaining issue -- it may be closed as Won't Fix, but at least it has been recorded.

This issue is now a duplicate of all of those other issues listed on #17 and here, and should be closed.