Migration path for domain

Already in the feature list. https://github.com/agentrickard/domain/issues/119

See CHANGELOG.txt.

We need someone to start working on this issue.

I'm working on source plugins for Domain Access affiliation migrations. This patch has a D7 source plugin for Domain Affiliations and Domain All Affiliates. Needs work! I'm in contact with another dev regarding the matter... hopefully they will help, too.

Hi folks!

I haven't write patch. I've made process sources.

rg_migrate.module:

<?php

/**
 * Implements hook_migration_plugins_alter().
 */
function rg_migrate_migration_plugins_alter(array &$migrations) {

  foreach ($migrations as $key => $migration) {
    /** @var \Drupal\migrate\Plugin\MigrationPluginManager $migration_plugin_manager */
    $migration_plugin_manager = \Drupal::service('plugin.manager.migration');
    $migration_stub = $migration_plugin_manager->createStubMigration($migration);
    /** @var \Drupal\migrate\Plugin\MigrateSourcePluginManager $source_plugin_manager */
    $source_plugin_manager = \Drupal::service('plugin.manager.migrate.source');
    $source = NULL;
    $configuration = $migration['source'];
    $source = $source_plugin_manager->createInstance($migration['source']['plugin'], $configuration, $migration_stub);
    if ($source) {

      $m = new RgMigration($migration);
      $process_domain_ids = [
        'plugin' => 'migration_lookup',
        'migration' => 'd7_domain',
        'source' => 'computed_domain_ids',
        'no_stub' => TRUE,
      ];

      // Users
      if ($m->id() == 'd7_user') {
        $m->setSourcePlugin('rg_d7_user');
        $m->addProcess([
          'field_domain_access' => $process_domain_ids,
        ]);
      }

      // Content
      if ($m->id() == 'd7_node') {
        $m->setSourcePlugin('rg_d7_node');
        $m->addProcess([
          'field_domain_access' => $process_domain_ids,
        ]);
      }

      $migrations[$key] = $m->raw;
    }
    else {
      throw new Exception('Error');
    }
  }
}

RgUser.php:

namespace Drupal\rg_migrate\Plugin\migrate\source\d7;

use Drupal\migrate\Row;
use Drupal\user\Plugin\migrate\source\d7\User;

/**
 * @MigrateSource(
 *   id = "rg_d7_user",
 *   source_module = "user"
 * )
 */
class RgUser extends User {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = parent::query();

    if (MIGRATE_DEBUG) {
      $query->condition('u.uid', MIGRATE_UIDS, 'IN');
    }
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = parent::fields();
    $fields['computed_domain_ids'] = $this->t("Domain ID");
    return $fields;
  }

  /**
   * @inheritDoc
   */
  public function prepareRow(Row $row) {
    $domain_ids = $this->select('domain_editor', 'de')
      ->fields('de', ['domain_id'])
      ->condition('de.uid', $row->getSourceProperty('uid'))
      ->execute()
      ->fetchCol();

    $row->setSourceProperty('computed_domain_ids', $domain_ids);

    return parent::prepareRow($row);
  }
}

RgNode.php:

namespace Drupal\rg_migrate\Plugin\migrate\source\d7;

use Drupal\node\Plugin\migrate\source\d7\Node;

/**
 * @MigrateSource(
 *   id = "rg_d7_node",
 *   source_module = "node"
 * )
 */
class RgNode extends Node {

  public function query() {
    $query = parent::query();

    if (MIGRATE_DEBUG) {
      $query->condition('n.nid', MIGRATE_NIDS, 'IN');
    }
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = parent::fields();
    $fields['computed_domain_ids'] = $this->t("Domain ID");
    return $fields;
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    $domain_ids = $this->select('domain_access', 'da')
      ->fields('da', ['gid'])
      ->condition('da.realm', 'domain_id')
      ->condition('da.nid', $row->getSourceProperty('nid'))
      ->execute()
      ->fetchCol();
    $row->setSourceProperty('computed_domain_ids', $domain_ids);

    return parent::prepareRow($row);
  }
}

Maybe this will helps somebody.

Awesome @super_romeo!

I forgot that the plug-in needs to extend Node. I will remake the patch.

How do you deal with the All Affiliates field?

Thanks for the domain editor User plugin. I will add that to patch as well.

Last... Do you have any sample YAML config to use as migration templates?

Thank you!!!!

How do you deal with the All Affiliates field?

Nohow yet...

Do you have any sample YAML config to use as migration templates?

I use hook_migration_plugins_alter(). Please see #5.

Thanks for the help @super_romeo!

I incorporated your code with some small changes into the patch. There's now 3 plugins in the patch file. Thanks for pointing out the module file code as well. Very helpful for anybody doing custom migrations.

For patching domain_access module to add support, I'm not sure we'll be able to use the hook_migration_plugins_alter() to do that. I think the easiest path towards approval would be using YAML in a new /migrations directory. I already have some of that started as well. I'll add that to a patch tomorrow.

This patch has 3 new, untested migration plugins:

NodeDomainAccess
NodeDomainAllAffiliates
UserDomainAccess

I will also start some testing soon.

Thanks again!

Tracking.

We should, in theory, add tests for this including a database fixture....

I forgot my RgMigration.php file:

namespace Drupal\rg_migrate;

class RgMigration {

  /**
   * @var array
   */
  public $raw;

  public function __construct(array $migration) {
    $this->raw = $migration;
  }

  public function id() {
    return $this->raw['id'];
  }

  protected function makePipeline(string $destination): array {
    $a = $this->raw['process'][$destination];
    $pipeline = [];
    // Simple field assignment.
    if (!is_array($a)) {
      $pipeline[] = [
        'plugin' => 'get',
        'source' => $a,
      ];
    }
    // Single element.
    elseif (isset($a['plugin'])) {
      $pipeline[] = $a;
    }
    return $pipeline;
  }

  public function addToPipeline(string $destination, $add_array) {
    $a = $this->makePipeline($destination);
    $this->raw['process'][$destination] = array_merge($a, $add_array);
  }

  public function addDependency(array $dependencies) {
    $this->raw = array_merge_recursive($this->raw, [
      'migration_dependencies' => [
        'required' => $dependencies,
      ],
    ]);
  }

  public function addProcess(array $fields) {
    $this->raw['process'] = array_replace($this->raw['process'], $fields);
  }

  public function addConstant(array $constants) {
    $this->raw['source'] = array_merge_recursive($this->raw['source'], [
      'constants' => $constants,
    ]);
  }

  public function setSourcePlugin(string $plugin) {
    $this->raw['source']['plugin'] = $plugin;
  }

}

I've used @matthands patch and wrote a patch which can migrate domain_access and all_affiliates from nodes and domain_access from users.

It can be used like this:

source:
    plugin: d7_node_domain_access
...
field_domain_access:
    -
      plugin: sub_process
      source: domain_access_node
      process:
        target_id: target_id
field_domain_all_affiliates:
    -
      plugin: get
      source: domain_all_affiliates

I added a function to the patch to migrate also the domain_source data. It's not the most beautiful code but it works so far.

Hi ,I have used comment 13 patch for domain access module.
I have run the migration using drush migrate-upgrade --legacy-db-key=migrate
but I still see only the content is migrated but they are not associated with any domain.
Domain data is not migrated.
Could someone help how to get this work?

@Drupalite1411:
The patch provides a new source plugin called "d7_node_domain_access". Update your migrations to use "d7_node_domain_access" instead of "d7_node".

I used the patch today to migrate Drupal 7 domain data. Everything works as expected.

Hi @ayalon,Thank you so much for responding.
I have placed migration "d7_node_domain_access.yml" in this location "\xampp\htdocs\migrate-project\web\modules\contrib\domain-8.x-1.0-beta6\domain\domain_access\config\install". Applied the patch given in the comment 13.Reinstall the module.
When I run the migration using "drush migrate-upgrade --legacy-db-key=migrate --configure-only", I can not see the d7_node_domain_access migration in the list. I can see all the migration generated but not the domain one.
Am I doing something wrong?

@Drupalite1411 try this:
1. Apply the patch
2. Generate your migrations with drush migrate-upgrade
3. If you have a node type page for example you should edit the generated migration like d7_node_page.yml by hand and change the source plugin:

source:
    plugin: d7_node_domain_access

For your domain fields add the following configuration:

field_domain_access:
    -
      plugin: sub_process
      source: domain_access_node
      process:
        target_id: target_id
field_domain_all_affiliates:
    -
      plugin: get
      source: domain_all_affiliates

4. Import config
5. Execute migration drush mim d7_node_page

Hello, I was checking in on the status of this patch as I see the status is in "Reviewed and tested by the community", was anyone able to use the current patch? Are there plans to port this patch or does it require more review and work?

I think it needs another review. We should also look into how to provide testing for this.

I've been trying patch #13
Patch installs ok
Generate our migrations with drush migrate-upgrade
Our 11 domains migrate ok with the upgrade_d7_domain.yml
Edit a node_complete_blah.yml as per #17
migrate-import node_complete_blah
```
SQLSTATE[HY000]: General error: 1364 Field 'sourceid2' doesn't have a default value: INSERT INTO "migrate_map_pa_node_complete_video" ("source_ids_hash", "sourceid1", "source_
row_status", "rollback_action", "hash") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_
4); Array
(
[:db_insert_placeholder_0] => 2b665b0353defac7fa029d84dde23dffffcbf62c29ce86702a4a909e8c448a0d
[:db_insert_placeholder_1] => 28453
[:db_insert_placeholder_2] => 3
[:db_insert_placeholder_3] => 0
[:db_insert_placeholder_4] =>
)
```
One node is imported with its appropriate Domain settings. The migration is now stuck with a status of Importing.

This particular migration (of node_complete_blah) ran properly before I'd installed Domain on the Drupal 9 site, so I know the rest of the migration is ok.
I've tried rollback and reset.

PS SOLVED

Turns out this happened because I'd already run that migration, so there's a migration map for it. Solution here

I went into the database and found there's a map table for every migration I've already done. So
'drop table migrate_map_pa_node_complete_video;'
then the migration imported fine.

Using patch #13 our nodes have the domain data migrated from D7, except for the Domain Source, which has come out as 'None' in all cases.

In D7 Domain Source is set in the domain settings Content Defaults as 'Use Active Domain'. I don't see Content Defaults in D9.

How do we get around that?

I appreciate this comment is fairly off-topic - but the people on this thread are likely to be the ones who could best answer - we need to migrate a drupal 7 Domain-based site to d9 - but in doing so we only need to migrate 2 of the 7 Domains - is it likely that we could customise the plugin with something like WHERE domain_id it X or Y or is 'all domains'.

Appreciate any help - I have someone with the skills to do this if it is feasible - just they aren't available currently and I need to try and form a plan for this site to avoid doing a bunch of minor tweaks in D7 that wouldn't be necessary post migration.

During migration, I tried the patch provided by @unkelhoebbi at #17. When I used this patch with my custom code migrations in "custom_module/migrations" folder it worked. However when I applied the same patch and used it for "configuration entities" generated in my "custom_module/config/install" folder, it didn't work. The Nodes are no more attached to the Domains I had assigned in D7 version of site.

No matter whether I use d7_node_domain_access or d7_node_complete, it still doesn't connects the nodes to domains in migrated database.
More details can be found at : https://drupal.slack.com/archives/C226VLXBP/p1693222680688409

Any suggestions ?

UPDATES :
After some more debugging and it was noticed, the Fields are correctly getting migrated with this patch. Not only fields but also field values are getting migrated correctly. However, the two fields viz., 'Domain Access' and 'Send to all affiliates' gets under "Disabled" section of "Form Display" of Content type automatically. It should follow the Drupal 7 settings for these two fields during migration to reflect in Drupal 8/9 after migration.

Is it something that patch needs to be worked on for fixing OR it's a problem of other migration plugins ?

Opened a MR and added three lines that avoid a warning when there is no source domain set

https://git.drupalcode.org/project/domain/-/merge_requests/51/diffs#03fe...

The code works for me migrating nodes. Given the support left for D7 I wouldn't bother adding any tests, that why I set it back to needs review

Just noting I used this patch from #26 (MR51) to successfully migrate node domain data into Drupal 10, with thanks.

I added this to the migration yaml:

id: custom_d7_node_domain_access
# Other migration yaml then
source:
  # Needs this plugin
  plugin: d7_node_domain_access
  # Example node type
  node_type: article
process:
  # Other node fields then domain fields
  field_domain_access:
      -
        plugin: sub_process
        source: domain_access_node
        process:
          target_id: target_id
  field_domain_all_affiliates:
      -
        plugin: get
        source: domain_all_affiliates
  field_domain_source:
      -
        plugin: get
        source: domain_source

and ran the migration with drush:

drush migrate:import custom_d7_node_domain_access