Problem/Motivation

One of the advantages of having migrations stored as templates is that they can be altered to fit the source data before they’re actually run. Before template support was committed, the only way to build dynamic migrations was with load plugins, which are very hard to grok and involved hijacking the Migration entity class and storage handler, which had undesirable side effects in addition to code bloat. Dynamic migrations are also hard to test — as far as I know, there’s no way to run them from the command line.

Proposed Resolution

I want to replace load plugins with builder plugins. They have the same goal — namely, generating migrations dynamically — but whereas load plugins acted at run time, builders run ahead of time, creating a set of migration entities from a template by merging the template with data from the source database. The generated migrations can then be saved, run and exported like any other migration. The main benefits of this are that build plugins are far simpler in every way than load plugins (and they involve less code), and the migrations they generate can be inspected and modified before run-time. Build plugins don’t need to hijack the Migration entity class or storage handler, so #2499173: migrate_drupal highjacks the Migration and MigrationStorage classes will be fixed as a matter of course.

Remaining Tasks

This is a fairly large change, so I think it should be applied in stages.

  1. Implement the builder plugin type in Migrate, and provide a few builder plugins in Migrate Drupal which replace the current load plugin functionality. Templates which currently use load plugins will also refer to the corresponding builders. Initially, all this will go in alongside the existing load plugin code, in order to prevent the patch from being too big and confusing. #2530030: Create the migrate builder plugin type
  2. Move absolutely all CCK field handling into dedicated cckfield plugins. There are already two issues for this: #2529144: Create cckfield plugin for text fields and #2528688: Create cckfield plugin for file fields.
  3. Remove load plugins and their infrastructure, Migrate Drupal’s overridden MigrationStorage and Migration classes, and the special interfaces used for load plugins’ CCK handling. #2549013: Remove load plugins
CommentFileSizeAuthor
migrate_generator.patch1.86 KBphenaproxima

Comments

phenaproxima’s picture

Title: Replace load plugins with generators » Replace Cthulhu-forsaken load plugins/specialize storage handlers with template processors

Renaming the issue for a funnier commit message.

phenaproxima’s picture

Title: Replace Cthulhu-forsaken load plugins/specialize storage handlers with template processors » Replace Cthulhu-forsaken load plugins/specialized storage handlers with template processors

Fixing a typo.

phenaproxima’s picture

Issue summary: View changes
Status: Postponed » Active

Unblocked. Full steam ahead!

phenaproxima’s picture

Title: Replace Cthulhu-forsaken load plugins/specialized storage handlers with template processors » [META} Replace Cthulhu-forsaken load plugins with migration builders
Issue summary: View changes

It's a meta-issue now, mofos.

benjy’s picture

Category: Task » Plan
benjy’s picture

Title: [META} Replace Cthulhu-forsaken load plugins with migration builders » [META] Replace Cthulhu-forsaken load plugins with migration builders
phenaproxima’s picture

Issue summary: View changes
phenaproxima’s picture

Issue summary: View changes
dawehner’s picture

TF @title

phenaproxima’s picture

Status: Active » Closed (fixed)

Load plugins are gone!

AND THERE WAS MUCH REJOICING. We have forsaken them, as Cthulhu before us. Praise be unto the tentacled one.