Suggested commit message:
Migrate in core #2125717 by mikeryan, chx, marvil07, bdone, jessehs, mpgeek, BTMash, fastangel, mongolito404, Moshe Weitzman, eliza411, YesCT, dawehner, cosmicdreams
Note to reviewers: you are doing yourself a huge disservice if you don't read https://drupal.org/node/2127611 the introduction handbook page first. It's short. And necessary.
The current upgrade path is unmaintainable, broken, a bad idea in the first place.
Instead of pointing a D8 codebase to a D7 database and try to bend each other together until they work together to upgrade to a D8 database use a migration process. Documentation has been started at https://drupal.org/node/2127611 -- I do not submit fundamental patches without some documentation written first.
With migration, you can utilize a running Drupal site to create configuration and content based on another datastore. The benifits of this approach are many
- We can evolve a datasource (of which a D7 database is just one kind) to a D8 datasource in a manner that doesn't destroy the original.
- We can make this system resilent to failure and resumable
- We can make this system friend to automated systems so that you can continuously migrate data from one source to another. So, migrate everything into a staging site and when going live have a minimal downtime to migrate since the last run. No more 24 hr downtimes to upgrade drupal.org!
- We can provide a flexible architecture that future migration efforts (from other datasources) can extend
- We can finally ship a capability to migrate data from D6, D7, and other systems into Drupal 8.
Focus of this patch
The current patch implements the migrate framework full of interfaces, one implementation for each, unit tests, a complete migration with a simpletest. For unit testing the database reading sources, an in-memory DBTNG Select driver is provided.
In future patches the foundational work here is not expected to change, we expect additions; possibly the actual migration entities and tests will be moved to another module so it can be disabled once it's done. That'll be much easier once we switch from a module-in-a-sandbox to a core-fork-in-a-sandbox.
The DBTNG driver will be extended to a full driver with Insert/Update/Merge support and moved under Database/memory.
Note to developers familar with the Migrate module
If you were familiar with Drupal 7 migrate module you will find very, very familiar pieces of code. We are not rewriting migrate, we are rearranging it into a D8 structure where migrations are configuration entities and almost every piece of logic is in a plugin: a source, a process, a destination or an id map plugin.
Past patch #1 we already have a lot more sources (practically all of D6 is covered, all of them unit tested). Each will be introduced with a real migration in a separate patch once we have the adequate destination too. These will be smaller patches, one bigger patch will be the batch of variables-to-config similar to the migration introduce here however that one will be next to trivial to review.
User interface changes
Not in this patch. Future work outlined here: https://groups.drupal.org/node/356283 describe our thoughts around providing a user interface. The UI is not a priority, by far, core will only ship with a very simple one, Bojhan offered help with this one.
We are adding a migrate API.
Additional impacts / effects
In implementing the migration system as a series of plugins for importing data from outside sources to internal systems we open the door to a great many data intensive applications. The source of the data simply needs to contain data relevant to Drupal. A complete migration system simply needs to provide a mechanism to extract the data from the source, transform the data to a form that Drupal understands and loads the data into Drupal. These are seperate and independent systems. In short, contrib space will have a very productive development cycle where this system can be made to solve many different data-oriented problems.