Comments

mikeryan created an issue. See original summary.

mikeryan’s picture

Ugh, not as easy as it may appear. On the source side, the only info we have tying a migration to its corresponding source module is the source_provider annotation on source plugins - this means we can't identify cases like search, where only configuration is being migrated from variables and there is no module-specific source plugin. And, not all source plugins specify the source_provider (node being a notable example - I'm not sure if there's a reason for this or if it's a bug). On the destination side, the 'provider' destination annotation has the same issue, plus for the entity:* destinations its not set according to the module implementing a given entity type. However, it does appear dependencies (not migrate_dependencies) gets set appropriately, so we may be able to use those - the question is when they're set, when creating the entity or saving it, the latter being unhelpful because we want to know before saving it.

I think, at least for the source side, we may want to add a module: key to the source configuration for all provided migrations. Something we'll need to do in core before we can make use of it here...

mikeryan’s picture

For now, I'm implementing the confirmation page using a hard-coded array listing the source module->destination module relationships for all current migrations in core - the idea is this at least lets us have a concrete UI to review and bikeshed. Once we've nailed down the UI, we can work out how it can get the data to drive it cleanly.

I ran into a bit of a problem in terms of identifying destination-side modules that may need to be enabled. In my D6 site, I have the book module enabled, but I don't have it enabled in D8. Ideally I'd like a message saying "hey, if you want to keep your book hierarchies you better go install the module". The problem, though, is that the d6_book template lives in the book module itself, and is not available unless the module is enabled - without the destination module enabled, we have no way of knowing that there is even a migration available for books. I'm not seeing a way around this, sort of injecting into MigrateTemplateStorage an alternate module handler with a getModuleDirectories() that scans all modules whether or not they're installed. Or, maintaining a laundry list of available migrations... somewhere... which is problematic in terms of discovery of contrib module migration paths. I had been thinking there should probably be a tool (in contrib) to analyze upgrade readiness, we may have to leave this problem to that tool.

mikeryan’s picture

Status: Active » Needs review
FileSize
22.77 KB

OK, here's the basic list of stuff being migrated.

webchick’s picture

Ok, we spent a good hour today going over this UI on a hangout.

It's important here to take a step back and look at the UI we're replacing. That UI is update.php. Update.php is not like a shining pinnacle of UX design, but it does accomplish the following things:

0) If something is up that's going to cause you problems down the road, it warns you about that right away and stops you from proceeding:

Error about not having access to run the script.

1) It first warns you about what the process you're about to engage in, and how to take proper precautions.

First screen explains the whys and wherefores and recommends backing up.

2) Next, it shows you what it's about to do and gives you a chance to review it before proceeding:

Shows two pending updates.

3) It performs operations in a batch...

Batch API shows 2 items processed

4) If there were problems, it tells you what happened, with something you can go and Google more to figure out...

Update #8005 failed, with error message.

5) You're able to review detailed logs to determine even more about what happened in order to help with said Googling...

In addition to update errors, shows some DrupalKernel errors which may be a factor.

6) Once you (think you) have fixed your problem you're free to attempt the update again. It will remember what things didn't work and allow you to re-try them:

Shows failed 8005 update again.

7) Repeat until you get a green board (see no more error messages) and then you know you are done:

No errors.

The current Migrate Upgrade UI is close on a lot of these points (esp. with the nice logging now, yay!), but it does miss a few. The TL;DR version is:

  1. I never know if I am done. The migration might've been successful, in that it shows no errors, but the key thing is that unless I'm sure that all of the modules on my old site have been accounted for, I'm not really done. I need to know what's missing so that I can go and download extra modules (e.g. php_code filter needs php module in contrib now).
  2. If there are problems, I can't try those again. Only recourse is to restore from backup and redo the entire migration.
  3. The two together == you can easily become very hosed. Making matters worse, if I mistakenly believe I am done, because it shows me some lovely green status message, and then I launch my D8 site and people start adding comments, nodes, etc. to it, and then 3 weeks later you're like "oh crap I forgot about the Donkey module" your only recourse is to restore from a 3 week old backup (which means data loss) or start futzing around manually with the DB tables (which means hiring Mike Ryan ;)).

For the first one, we talked about tracking the source modules from the old database against the target modules in the new database (the existing patch has some bug which makes some of those cells blank, but that general idea). If there are source modules unaccounted for in D8, raise an error condition. This gives me the opportunity to remember "Oh, crap, right, I totally installed Advanced Poll module that one time. Here, let me uninstall that from my D7 site so it's not crufting up my DB."

For the second one, we talked about consulting the migrate_map* tables and allowing re-doing the migration multiple times, skipping rows that already migrated fine. This would match what most people are going to want to do, which is do the whole shebang upgrade in at month X, then spend the next little while fixing whatever problems, and then do a final "pick up whatever's missing, plus the new content/users/etc. since then" just before the big switch-over.

Mike said both of those sounded doable, so he's going to work in that direction for now before we get to the final round of nitpicking labels, etc.

mikeryan’s picture

I've created some child issues to #2562849: [meta] Confirmation page to address follow-up details around the confirmation page - my plan is to commit the basic list of modules being migrated (or not) under this issue, then refine in the further issues.

Issues around restarting/continuing the upgrade path will fall under #2567119: [meta] What to do when upgrade has already been done?.

Thanks!

  • mikeryan committed f83374a on 8.x-1.x
    Issue #2565159 by mikeryan: Create basic confirmation page
    
mikeryan’s picture

OK, committed what I have so far (source modules without migrations show "No upgrade path available"), to be refined in follow-up issues.

mikeryan’s picture

Status: Needs work » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.