I followed this guidance https://www.drupal.org/node/1819738 to extend the d2d DrupalNode6Migration class:

class CadeMigrateFields extends DrupalNode6Migration {

  public function __construct($arguments = array()) {
    parent::__construct($arguments);
  }

  protected function query() {
    // Get the default query 
    $query = parent::query();

    // Get OG stuff as per: https://www.drupal.org/node/2033293#comment-9371471
     
    $query->leftJoin('og_ancestry','oga','n.nid = oga.nid');
    $query->addExpression('GROUP_CONCAT(DISTINCT og.group_nid)', 'og_group_ref');       

	$query->addField('oga', 'group_nid', 'gid');
	$query->addField('oga', 'is_public', 'is_public');

    return $query;
  }
}

I added this to my files[] in cade.nfo as an include.

hook_migrate_api is rather simple:

/**
 * Implements hook_migrate_api().
 */
function cade_migrate_api() {
  $migrations = array();
 
  $api = array(
    'api' => 2,
    'migrations' => $migrations,
   );
   
   $common_arguments = array(
    'source_connection' => 'for_migration',
    'source_version' => 6,
    'group_name' => 'cade_group',
    );

    // No need to register the CadeMigateFields since it only extends DrupalNode6Migration.  Or, should I?
  
  return $api;
}

Is there anything else I need to do to make Drupal d2d see my CadeMigrateFields class?

Can someone please help?

Comments

SomebodySysop’s picture

Issue summary: View changes
SomebodySysop’s picture

Title: Migration could not be constructed » Migrate d2d doesn't see my class which extends DrupalNode6Migrate
Issue summary: View changes
SomebodySysop’s picture

Title: Migrate d2d doesn't see my class which extends DrupalNode6Migrate » How to get Migrate d2d to see class which extends DrupalNode6Migrate?
eneko1907’s picture

The $migrations array would have to be populated.

I'd remove the empty $migrations array, and instead initialize the array right there, like this:

 'migrations' => array(
      'CadeFields' => array(
        'class_name' => 'CadeMigrateFields',
      ),
 ),

Then please declare/register your file containing the class CadeMigrateFields in the .info file of your module (unless is all bundled in one big php file.. not sure how you structure this migration.

eneko1907’s picture

I added this to my files[] in cade.nfo as an include.

not sure what you mean there (include?), but let's say your "cade.info" file would have a line like this:

[...]
files[] = CadeFieldMigration.php
[...]

.... and the CadeFieldMigration.php is right in the same folder "cade" and it looks like what you quote in your first block, seems correct...

SomebodySysop’s picture

Thank you very much! These are the errors that come back when I implement your suggestions:

  • Notice: Undefined index: destination_type in DrupalNodeMigration->__construct() (line 43 of /var/www/html/websites/drupal/master/sites/all/modules/migrate_d2d/node.inc).
  • Notice: Undefined index: source_type in DrupalNodeMigration->__construct() (line 44 of /var/www/html/websites/drupal/master/sites/all/modules/migrate_d2d/node.inc).
  • Migration CadeFields could not be constructed.
  • No source_version provided in migrate_d2d migration arguments.

I am running the migration using the d2d ui. The source info is entered when I click on "Import from Drupal". Is there any way to write the code so that it will use the source provided to DrupalNode6Migration?

This is what I put in hook_api:

/**
 * Implements hook_migrate_api().
 */
function cade_migrate_api() {

   $migrations = array();
  
   $migrations['CadeFields'] = array(
        'class_name' => 'CadeMigrateFields',
   );

  $api = array(
    'api' => 2,
	'migrations' => $migrations,
   );
     
  return $api;
}

eneko1907’s picture

I am not seeing the full picture, but I am guessing the $common_arguments are not being picked up by your class.

You may want to add to your CadeMigrateFields class some $arguments to help with the context it may need.

May be something like this:

class CadeMigrateFields extends DrupalNode6Migration {

  public function __construct(array $arguments) {
   $arguments += array(
      'description' => '',
      'source_connection' => 'for_migration',
      'source_version' => 6,
      'source_type' => 'story',  // here goes the machine name of your source D6 content type, replace as needed
      'destination_type' => 'article', // here goes the machine name of your destination D7 content type, replace as needed
      'user_migration' => 'CadeUser', // assuming you have a "user" migration.. otherwise, omit this...
    );
    parent::__construct($arguments);
  }

  protected function query() {
[...]
SomebodySysop’s picture

You are right. I was not passing common arguments along because they cause even more errors.

Here's what I'm trying to do:

When I run the migrate_d2d, it asks me which content types I want to bring over. I select the content types, then tell it to save settings. It then creates a list of content types to migrate that I can review.

When I look at a content type, I see the query that will be used to get all nodes for that content type. This query is created by the DrupalNode6Migration class. I need to add two more fields from another table, thus the leftJoin in my code above.

I do not wish to create another node migration class. I wish to modify the query created automatically by migrate_d2d using the DrupalNode6Migration class. That is what this document appears to say I should be able to do: https://www.drupal.org/node/1819738

My problem is that I don't seem to be able to get migrate_d2d to recognize the class I created above. You gave me some insight on how to do that, and that's where I am now.

I actually tried using common arguments before. The problems with seem to be:

1. I don't know the name of the source_connection. It is created dynamically by migrate_d2d.
2. I know the source_version = 6.
3. I want this code to be used for all content types in the d2d migration, otherwise, I have to create a new file for every content type I wish to migrate.

What I believe I need to do is get the arguments that have been created by the migrate_d2d ui. But, I don't know how to do this.

But, I will try your suggestion and see if I get a better result.

SomebodySysop’s picture

I modified my class as follows:

class CadeMigrateFields extends DrupalNode6Migration {

  public function __construct(array $arguments) {
   $arguments += array(
      'source_connection' => 'for_migration',
      'source_version' => 6,
      );
    parent::__construct($arguments);
  }

  protected function query() {
    // Get the default query 
    $query = parent::query();

    // Get OG stuff as per: https://www.drupal.org/node/2033293#comment-9371471
     
    $query->leftJoin('og_ancestry','oga','n.nid = oga.nid');
    $query->addExpression('GROUP_CONCAT(DISTINCT og.group_nid)', 'og_group_ref');       

	$query->addField('oga', 'group_nid', 'gid');
	$query->addField('oga', 'is_public', 'is_public');

    return $query;
  }
}

And I now get these errors:

  • Notice: Undefined index: destination_type in DrupalNodeMigration->__construct() (line 43 of /var/www/html/websites/drupal/master/sites/all/modules/migrate_d2d/node.inc).
  • Notice: Undefined index: source_type in DrupalNodeMigration->__construct() (line 44 of /var/www/html/websites/drupal/master/sites/all/modules/migrate_d2d/node.inc).
  • Notice: Undefined index: description in DrupalMigration->__construct() (line 136 of /var/www/html/websites/drupal/master/sites/all/modules/migrate_d2d/migrate_d2d.migrate.inc).

Now, I suppose I could try filling these in (description, source type, destination type, etc...), but that means I'd need to have a different class for every content type, which sort of defeats the purpose of "extending the DrupalNode6Migration" class, which is what I am trying to do.

This is from the "extending Migrate d2d Classes" document here: https://www.drupal.org/node/1819738

Another common use case for overriding migration classes is modifying the base query. Each of the migrate_d2d classes has a query() method, returning the query object used to pull source data. By doing this rather than hard-coding the query into the constructor, you have the opportunity to customize the query for your application:

Is there some way to pass the common arguments currently being used by the class we are extending (i.e., DrupalNode6Migration)?

Or, is there some way to declare this as an extending class and not a migration class?

Thanks so much for the assistance!

eneko1907’s picture

I see, thanks for the clarification -

I was under the impression that you were doing this in code, not using the d2d UI. I get you are mixing both methods.

The guidelines you quote in three separate comments in this issue are for extending the d2d classes.. in code. What you want can certainly be done -- the better way to combine both methods may be explained by the migrate_d2d maintainer at https://www.drupal.org/node/2118243#comment-8715933 .

I wonder whether using the D2D UI is the best approach for your case - I am really not sure - Writing your own little module that extends the Node 6 Migration class for your migration is not much more code that what you already done - the maintainers have done the heavy lifting for us.

SomebodySysop’s picture

Thank you very much for your insights. You may very well be right. By now, I probably could have written my own migration class. I know that I could easily modify the migrate_d2d inc file for DrupalNode6Migration to do exactly what I want.

But, as you point out, the UI does a lot of heavy lifting, and being able to extend classes that do most of what we want (rather than writing our own from scratch) is a very useful capability for people like me who are totally new to migration. And, for the sake of the overall community, someone needs to explain how extending classes documented here https://www.drupal.org/node/1819738 are registered.

The link you provided above seems like it might work. Going to give it a try. Again, many thanks.

eneko1907’s picture

You are welcome.

Writing your own class (and module) is simpler than you describe here. IMO, you do not really need to modify the migrate_d2d, you basically extend the class DrupalNode6Migration as you do up there. Then you would have to do a bit of the standard drupal module work with the .info, and specifically, in a possible .migrate.inc (.module OK), you would instantiate the hook_migrate_api. This is what I meant when saying the heavy lifting is done for us, not just in the D2D UI, but in the D2D code as well. For what I see, you already done half of that work up there!

However, what Mike Ryan seems to be saying in that thread (BTW, please do not re-open a closed issue, make a new one or stick with this) is that, in order to pick it up where the Migrate D2D Wizard left, you may want to use the hook_migrate_api_alter to introduce your mods to the query object (or whatever you need to do that the UI is not really equip to do as of now). Again, this proposed cade_migrate_api_alter will imply writing a small module, which is what seems the thing to avoid here :). I get it, it would be best if you can get all the way there w/o writing one line of code. My preference too, but not quite there yet (OG, etc)

SomebodySysop’s picture

Thanks. Sorry, I'll close that issue. I just really think this needs to be better documented.

I'll report back here whether hook_migrate_api_alter works does the trick. In fact, if it works, than I'll document it.

eneko1907’s picture

Migrate sometimes feels like so much more than migrate, doesn't it?

I think most if not all the documentation for the migrate_d2d is 'there', including the class registration you are wondering about.

I think the whole documentation is a bit scattered: There is quite a bit of very nicely structured documentation linked in the "Documentation" project page link, there is some geeky docs at the contribAPI site, there are several examples including the beer.inc and wine.inc (and many more), there are blog posts, drupalCon videos on migrate (Drewish, Mike Ryan, etc - since Denver-2012) and then there is the issue queue.

In essence, the documentation is 'all' there, but certainly at times it feels like what you need is not there, and well, it may feel like a fishing expedition.

Your envisioned contribution to the docs may help a lot - it will be all more than welcome!

SomebodySysop’s picture

Looks like I'm a little closer. I found this https://www.drupal.org/node/1813672 where the poster is essentially trying to do the same thing I am. The moderator, mikeryan, essentially suggests the same solution using $query = parent::query() -- same thing I'm trying to do.

So far, so good. Next, there is this bit of code I located which everyone seems to agree is used to declare the extending class:

  // which will be unique for each migration you register.
  $user_arguments = $common_arguments + array(
    'description' => t('Migration of users from Drupal 6'),
    'machine_name' => 'User',
  );

  // We just use the migrate_d2d D6 migration class as-is.
  Migration::registerMigration('MyUserMigration', $user_arguments['machine_name'],
                               $user_arguments);

Going to try working with this to see where it gets me.

mikeryan’s picture

Component: Code » Documentation

Extending UI-generated migrations using custom migration classes does need to be documented.

SomebodySysop’s picture

True. I realize that now after going in circles for several days. But what would be helpful is a definition of is what is meant by "extending" when you say "Extending migrate_d2d classes".

For example: I run migrate d2d ui and save settings. I go to dashboard, click on group, get a list migration tasks. I click on a listed node task, then click on "Source" for that node migration. I see a query.

I need two fields added to this query. So, I figure I can use "Extending migrate_d2d classes" code to "extend" this query so that the next time I run this migration, go to the dashboard, locate the same node task and look at it's source, the two fields I need will be included in the query and available to map.

To me, and I suspect to others, that is what "Extending migrate d2d classes" means. But, that is NOT how this works.

The way it works: You have to create a new migration class. Just follow existing documentation on how this is done. If you want to use, for example, the node query from the DrupalNode6Migration class, then you create the class with class MyNewClass extends DrupalNode6Migration class Then, you insert into your MyNewClass:

  protected function query() {
    // Get the default query 
    $query = parent::query();
    ...
   }

Now, you can "extend" this query to be used in your new Migration class, which now should also appear in the Migration dashboard -- but in a different migration group from that created by the migrate_d2d ui. And, unlike with migrate_d2d, you have to create a new class for every content type you wish to migrate.

And this is fine, this is great, this is useful.

But, I wouldn't call it "extending ui-generated migrations" since what you are really doing is creating a new migration using a query from a migrate_d2d class.

I guess it's just a matter of semantics. I was confused because I didn't know what I was doing. My bad.

But, if I could make one suggestion: It would be nice if there was a way to pass the arguments dynamically generated in the migrate_d2d ui and pop them into our "extending" class. Then we'd really be "extending" as opposed to re-creating.

mikeryan’s picture

When we speak of extending a class, that's in the normal PHP sense - MyClass extends OtherClass.

It would be nice if there was a way to pass the arguments dynamically generated in the migrate_d2d ui and pop them into our "extending" class

If you've followed https://www.drupal.org/node/2118243#comment-8715933 to substitute your class for the default class, then it should have exactly the arguments the UI configured on the default class.

SomebodySysop’s picture

Status: Active » Closed (fixed)

Beautiful. I still have to test this, but you have certainly answered my question. Furthermore, I have to rollback and start this migration all over again. So, I'm closing this issue. I am better informed now and at least have an approach to resolve most, if not all the problems I am currently having.

Thanks for your help.

own3mall’s picture

How do you inform D2D UI of your extended classes? Where do you save them? Do you save them for example in sites/all/modules/migrate_d2d/mycustom.inc? Do you save these classes anywhere and then just add them to the files[] array in sites/all/modules/migrate_d2d/migrate_d2d.info so they are run when performing the migration?

More documentation is needed!

RedEight’s picture

own3mall, it's fairly straightforward to inform migrate_d2d of the class. You do the same as you would for Migrate API: in hook_migrate_api set the class_name of your migration to your custom class. Then in the .info file, add a files[] line that points to whatever file contains the class definition. If it isn't showing up on the migration page, be sure you clear cache.