Problem/Motivation

We need to be able to import data from D6/D7 sites.

We will need different migrations/sources for D6/D7, possibly even for 7.x-1.x and 7.x-2.x due to the rather different data structure. We can also write a source plugin that supports both, though.

Proposed resolution

Start with one version. Define a source and a destination class and a migration config entity. We need at least a different migration for subscribers/subscriptions and newsletters. We might also want to separate subscribers and subscriptions, depends on how complicated the data processing will be.

There will be at least two sources as written above, both should provide the same data structurs, so we can use it with the same destination.

Remaining tasks

User interface changes

API changes

Data model changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir created an issue. See original summary.

Arla’s picture

Assigned: Unassigned » Arla
Arla’s picture

FileSize
2.18 KB

To my understanding, the migration mechanisms are still experimental. I'm not really sure how they are intended to be used. But this is a start.

To perform the migration, install migrate_drupal (in core) and migrate_tools (part of migrate_plus), and run something like drush migrate-manifest modules/simplenews/simplenews.manifest.yml --legacy-db-url=mysql://foo:bar@127.0.0.1/drupal7.

Newsletters are not yet supported, only subscriber entities.

Arla’s picture

Moved to migration_templates/, added newsletter. Drush usage remains the same.

Subscriptions and blocks are still to do.

Arla’s picture

Subscriptions are now basically migrated, but the target_id of subscriptions are exactly like newsletter_id, i.e. not considering the migration changes.

Annotated screenshot

Arla’s picture

Status: Active » Needs review
FileSize
6.23 KB
4.51 KB

Fixed the target_id processing. Then did some cleanup.

Now ready for review. So this migrates from 7.x-2.x to 8.x. I haven't even checked 7.x-1.x nor 6.x (yet) so I don't know what's different.

Check #3 on how to test this. I removed the manifest file, but you can easily create it as in the previous patches. I'm having problems with Xdebug+Drush, that's why I created the test controller in #5.

Status: Needs review » Needs work

The last submitted patch, 6: migrate-2596243-6.patch, failed testing.

Berdir’s picture

+++ b/src/Plugin/migrate/source/d7/Newsletter.php
@@ -0,0 +1,67 @@
+  public function getIds() {
+    return ['newsletter_id' => ['type' => 'string']];

Isn't this an integer in 7.x? (type: serial)

Looks quite nice, but why the switch back to optional config instead of templates? Note that you're using old instructions, see https://www.drupal.org/node/2257723.

With the optional config, it then explodes on the missing database connection.

Arla’s picture

Status: Needs work » Needs review
FileSize
6.22 KB
1.46 KB

Well, it's tricky to find documentation on migration, and to know which pages are more up-to-date and correct.

Switched back to templates and the 'serial' type.

Arla’s picture

Sorry, the patch in #9 contains a change to the subscriber template which doesn't seem to work I expect. Here's the one that should work.

Berdir’s picture

Ok, this works pretty well with 7.x-2.x.

To test, I went first to d8/upgrade, then went until the confirmation step. It reports that it can't find migrations for simplenews but that's just because the information there is hardcoded and it doesn't understand what belongs to which module.

Then, I used drush ms to see the list of migrations it created and drush mi d7_simplenews_newsletter/d7_simplenews_subscriber to run the specific migrations.

I think we can commit this as a first step, with a follow-up to also support 7.x-1.x (the biggest difference is that newsletters are a mix of terms + a custom table, so the source needs to detect that and run two different queries, ending up with the same data) and 6.x-* (same structure as 7.x-1.x, more or less, but different table names for term tables afaik).

  • Berdir committed edbeaa3 on 8.x-1.x authored by Arla
    Issue #2596243 by Arla: Define migrations for simplenews subscriber and...
Berdir’s picture

Status: Needs review » Active

Committed, lets extend this to cover 6.x and 7.x-1.x too!

bisonbleu’s picture

Hello, I have to migrate a website from Drupal 6 to Drupal 7 for a client. This website uses Simplenews. I've successfully used migrate and migrate_d2d for importing all content type data.

I'm not exactly sure how to proceed to migrate Simplenews data. This thread seems to address this issue but for Drupal 8.

Any ideas how I can do this?

Thanks!

hussainweb’s picture

Created #2802227: Define migrations for data from simplenews 7.x-1.x for migration from 7.x-1.x version of the module.

jonathanshaw’s picture

Title: Define migrations for simplenews subscriber and newsletter data » Migrations for data from 7.x-2.x
Related issues: +#2802227: Define migrations for data from simplenews 7.x-1.x, +#2917919: Define migrations from D6

Now that this has been committed to 7.2, closing in favour of #2917919: Define migrations from D6 and #2802227: Define migrations for data from simplenews 7.x-1.x

rwmanos’s picture

While migrating using drush migrate-upgrade a Drupal 7.56 site to Drupal 8.4.4 with simplenews-8.x-1.0-alpha3 installed, the following error occurs:

Upgrading d7_simplenews_subscriber
Calling Drupal\migrate\MigrateExecutable::import() [123.82 sec, 140.95 MB]                                    [debug]
The "serial" plugin does not exist. [123.82 sec, 141 MB]                                                  [error]
Returned from hook drush_migrate_upgrade [123.82 sec, 140.95 MB]                                              [debug]

and the migration is interrupted.

jonathanshaw’s picture

@rwmanos please create a new issue; you may also get better help on Drupal Answers.

Arla’s picture

Assigned: Arla » Unassigned
AdamPS’s picture

Status: Active » Fixed

This issue was fixed in #12. There are separate issues for follow on tasks as described in #16.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.