Problem/Motivation

We will eventually need to migrate Drupal 8 to Drupal 9 (and possibly Drupal 10).

Also while it's less common, there are use-cases for migrating Drupal 8 to Drupal 8 - things like migrating a multi-site to domain module.

Proposed resolution

Add Drupal 8 source plugins, and test coverage for an 8-8 migration.

This will provide the basis for the 8-9 migration path, since these would only change to due to changes in the migrate API in 9.x

If we add this before opening 9.x then we have the potential to keep 9.x in a 100% shippable state with support for 6/7/8-9 migration paths (even if we might move the 6.x sources to contrib at that point).

This is a much lower priority than 6.x/7.x sources, so marking postponed.

Remaining tasks

User interface changes

API changes

Data model changes

Comments

catch created an issue. See original summary.

jian he’s picture

I could not wait for the 8-8 migration, and do not know how to implement it.
Is there any 8.x sources as an example? such as node or user source.
If there has an example, so we can study it and coding the migration for other modules.

mikeryan’s picture

No work has yet begun, or is anywhere near beginning, on D8 sources - we're still busy nailing down the D6 and D7 migration paths, hence this is a Postponed Plan rather than an Active Task. My expectation is that we'll do this initially in contrib (like the D5 migration support) with an eye towards merging into core in a later 8.x release.

mikeryan’s picture

Note that while the migrations themselves, and narrow source plugins for specific cases, will most likely be developed first in contrib, I think a configuration source plugin is broadly useful enough (and a necessary enabler for the contrib work) to go into 8.1 core: #2621500: Configuration source plugin.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xjm’s picture

Status: Postponed » Active

This can be active for the next development minor (currently 8.2.x).

xjm’s picture

Priority: Normal » Major

Also, I think this is major.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

quietone’s picture

If we follow the existing pattern, we will also need a D8 test fixture. Or even better, more than one, to cover different use cases.

Irregardless of how many there are, I think they need to be built via the UI with the intention of making them once and only once. I mean, with all the modules, settings, data and modifications needed. We should not have to modify the dump for each and every migration. It is time consuming and prone to error. The dumps should only need minimal attention, such as updates.

And probably improve the tools, so that things we test, like timestamps, don't get changed every time a patch is made after using the UI. Or to automatically change text, like the maintenance text, to begin with 'Drupal 8' so we can confirm it migrated to the new destination correctly.

chx’s picture

8-to-8 should replace the exception thrown in SqlContentEntityStorageSchema::onEntityTypeUpdate

xjm’s picture

Title: 8-8/8-9 migrations » D8-D8 and D8-D9 migrations

(Just trying to fix that this issue has about the most unsearchable title.)

chx’s picture

Is this a challenge to find an issue with a more unsearchable title?

jibran’s picture

Title: D8-D8 and D8-D9 migrations » Migrations from Drupal 8 to Drupal 8 and Drupal 8 to Drupal 9
Berdir’s picture

We started working on something for a project we have, put it on d.o now: https://www.drupal.org/project/migrate_drupal_d8

Not extensively tested yet, we're still developing it.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mikeryan’s picture

Priority: Major » Normal

Since the plan is that Drupal 9 will basically be Drupal 8.x with deprecated code/features removed, it seems we won't need an upgrade path to Drupal 9, right? The pain of migrating to a new Drupal instance to achieve an upgrade should be a one-time thing (or at least, not something we'll need to do again for a couple of major releases). So, while it will be good to have D8 sources to support custom D8->D* migrations, I don't think that rises to Major priority.

xjm’s picture

Priority: Normal » Major

Oh, this is definitely major. The whole policy relies on having D8 sources that are part of the code that gets deprecated and upgraded. Finishing D6/7 migrations of course needs to come first, but this is a key part of making upgrades easier in the future. Edit: already we hear from people that being able to do migrations within D8 would save a lot of pain for experimental modules and so forth. We had a meeting a couple weeks ago with experimental module owners and one of the key takeaways was a need for migrations.

Berdir’s picture

That's an interesting topic that goes far beyound this issue I think.

Pure API deprecations are fairly easy, we have a new API, both do the same we eventually remove the old, done.

Deprecating configuration or storage is a lot more complicated. I honestly have no clue yet how we're supposed to that. The configuration of rest.settings to configuration entities was one example that we've already done in 8.x and honestly it was/is a mess, broke modules like rest_ui and so on. If we'd been strict/honest about BC in rest.module, that would have never been allowed to happen the way it did. (Not blaming anyone, but that's how it is).

We'd need a sync in both ways, so that the data/configuration can be found in both the old and the new place, but what if the new structure can't even be fully representated in the old structure? Or if it's a change that by defintion conflicts with the old way? so many "interesting" problems.

But I'm not sure if we need migrations because as written, the data needs to be in both places from the beginning, so the only thing would be to automatically run migrations in update functions and I doubt that is easier than just writing code directly.

xjm’s picture

Right, sorry, the bit about migrations within a D8 site for experimental was just an aside. The main point is that D8 sources are a requirement for the new upgrade and deprecation model to actually work, because otherwise we're stuck with legacy database tables forever again and are back to the pain of D6 upgrades. The expectation will still be that you create a new D9 site and migrate into it; the change is just that you don't have to wait 1-2 years for contrib modules or rewrite all your custom code. Migrate is an essential piece of the puzzle for us.

mikeryan’s picture

Does this mean that the expectation is if we have a BC change in data model (database table), at the point we implement the new model we leave the data represented in the old model until D9? My expectation was that data would be upgraded in place as we go if we have such an instance - as Berdir suggests, maintaining two data models at once seems painful.

I think it may be hard to picture how this works in practice until we hit a concrete example (are there any such changes in the works now?).