Migrate data from 7.x-1.3 to 8.x-2.x

Yeah, it's on my to-do list, particularly since the migration paths are the primary thing holding back an 8.x-2.x release. I haven't worked on it yet because the 7.x-2.x data structure is not finalized, but I suppose there really isn't any reason I can't write the path for 7.x-1.x. I'll look into it sometime this weekend.

Well, if you get to it before I do there is a 7.x-1.3 data file for creating tests already in the 8.x tests directory. I'm not sure if that will be reusable for 8.x migrations. It was originally made for testing 7.x-2.x upgrades.

Testing the 7.x-1.3 ingredient migration.

It helps to set the status.

Added the ingredient settings migration, not including the ones that got changed to field settings. The test is failing locally even though the migration works in practice. I want to see what it does here.

I thought I'd copied in the fixture path method. Apparently not.

Updated migration and test namespaces to match the Recipe source version.

Quitting for the night - just uploading what I got done. I added all of the recipe fields. It worked in manual testing. I didn't finish the automated tests before deciding to stop. Another migration to copy the remaining ingredient settings into the recipe field still has to be written, but I think that's the last thing.

That's fine. I think I named it that to match the version of the module that the data came from, but it doesn't really matter. It will reduce confusion.

If you still have your first patch you can upload it, if you have any interest in me reviewing it and telling you why I did things differently.

Your comment in #13 suggests that the recipe field migrations from #11 did not work for you. Did it not work?

Huh. Weird. Maybe I forgot to include something in the patch. I was getting tired at that point. I'll check in a few minutes when I'm back on my laptop.

The recipe data migration files are in #11. Check to make sure you have these two files:
- migrations/recipe713_recipe.yml
- src/Plugin/migrate/source/recipe713/Recipe.php

If you don't, then maybe you accidentally downloaded the wrong patch. If you do, then I'm not sure what went wrong, but it would be a good idea to find out. Maybe you just needed to rebuild your cache before the upgrade. I had that happen once, but it was because I exported the base D8 database so that I could just import that after every test instead of going through the install process. It doesn't make sense that you would have been able to successfully import the ingredients, but not the recipe fields if you were experiencing the same issue.

When you go into the migration path confirmation form, you should see Ingredient and Recipe listed together under the destinations. If you know you have the files, but Recipe isn't listed there then something is wrong and the recipe data plugin isn't being detected.

RE: fixtures
I encountered the exact same error about the tables not being installed when writing the Ingredient module tests. D8 fixtures must have different requirements than ones made for D7. Since I already had core's D7 fixture (located in the migrate_drupal module) open to see what a newer fixture looked like, all I did was copy the create table commands for the system and variable tables from it into the ingredient fixture. It worked with no problem.

RE: non-migrated recipe data
Well, that's just disappointing. It looks like not only did your ingredient references not get migrated, nothing else did either. At the moment I don't have any idea what the problem is. I ran one test migration with the code in that patch last night. It contained two test recipes (I no longer have access to real-world data). Both had all their fields imported with no issue. I'll think about it, run the migration again, and see if the test has the same issue that you did.

I'm also inclined to think that the image issue was unrelated. The recipe imports run separately from the main node migration and their D7 fields. Maybe if the node migration was marked as being failed, that might cause it. The recipe data migration is dependent on it. Also, because it's separate it won't be listed as "Nodes (Recipe)" in the log. You want to look for the "Recipe data" migration log, which is the title given in the migration YAML file.

I had a thought. Have you checked your database to see if the field data was actually migrated? If the data got migrated then maybe it's just not showing up on the front end. I expect that the nodes are given the same ID numbers that they had in the D7 site, but what if they are being changed during the migration? The fields are being given the D7 nid. If the nid's are being changed for some reason then the nid in the recipe migration simply needs to be run through the migration processor plugin to map the old nid to the new one.

The dependencies of the recipe migration and of the recipe nodes caused the number of tables that had to be installed by the test to balloon until I just got sick of it and said "Screw it, I'll just install core's fixture, wipe the node tables, and then install the recipe fixture after." So, that's what I did. This wasn't necessary in D7 and I'm hoping that there's a better way in D8, but I didn't get around to looking into recommended methods for creating fixtures and maybe not having to load quite so many tables. So this is what I have for the moment, which seems to be working.

There were a couple of other minor changes in that patch that I'd forgotten to include with git add: a migration constant that I'd added, but didn't need; an update to the test docblock; and changing the test method name to assertRecipeFields().

Here's an interdiff that includes those changes, if you're interested.

I recreated the fixtures using the D8 fixture-creating tool. The recipe713 fixture has to include more unrelated tables than I would like, but it does make the code simpler.

I wasted two nights trying to figure out what wrong with the ingredient field setting migration only to figure out that it was... nothing. Nothing was wrong. I had just failed to set the value that I expected in the D7 instance from which I was migrating.

Added the ingredient field display settings migration. This should complete the migration path from 7.x-1.3. Please check it out, @mparker17. We still need to figure out what was going wrong with the recipe data migration in your case.

Don't worry about it. Thanks for checking it.

Could you dump your node and recipe tables and upload them?

Thanks for reviewing it and for figuring out what was going on. Unfortunately, this is concerning to me and I think it's probably a major bug with the upgrade path. The D7 recipe data have no language other than what their nodes are. I think that the node's language needs to be inserted manually into the D8 fields in order keep problems like this from happening. I'm going to work on it a bit more and try to resolve the issue.

This was a challenge to figure out, but fortunately I was able to follow along with what was done for multilingual nodes. The differences in how D8 handles translations versus how D7 handles translations are at the heart of the issue. In D7 all nodes get separate nid's and translations just reference the original. In D8, a piece of content and all of its translations are treated as a single node with one nid.

@mparker17, if you're still watching this issue and you still have access to unmigrated data could you try out this patch to see if it properly migrates your content?

Take your time. I just appreciate that you're willing to test it.

Don't worry about it. Thank you for the review.

Thanks again for the help, @mparker17.