I'm working on a D6->D7 migration, and I'm running into an issue where certain CCK fields are not imported into my D7 site, even though the mapping has been specified in my class constructor and the Migrate UI screens show the mapping as properly set up. After debugging,looking in my prepareRow() method, and combing through lots of code (including D6 CCK code), I think I've found the problem; migrate_d2d does not properly migrate from CCK fields that are shared among multiple content types. The TL;DR version is that it chooses for some reason not to join on content_field_* tables for commonly shared fields.

For those who don't understand what I mean, in D6 and older, if you create a custom CCK field and only add it to one content type, the columns for that field are put in the same table as the content type (content_type_*). However, once that field is used in multiple content types, a separate table is created for it (content_field_*), and the columns for that table are removed from the content_type_* table. This means that if you want the data for that field, you have to join on the content_type_* table to the content_field_* in nid and vid (this is corrected in D7 in that all fields get their own custom table - field_data_* - regardless of whether or not they're only used in one content type).

The code that is causing this is in the various query() methods (e.g. DrupalNode6Migration::query(), DrupalEntity6Migration::query(), DrupalVersion6::getSourceValues()):

      foreach ($field_info as $field_name => $info) {
        if (isset($info['columns']) && !$info['multiple'] && $info['db_storage']) {

The code only goes into that loop if db_storage == 1, and there is no else statement. The problem is, db_storage = 1 (in the content_node_field table) only for fields whose data is stored in the content type base table. Therefore, the join is never added to the query, and the data for that field is never obtained.

My fix for this is to add a query() override in my migration class to get this data (this is for a CCK field called field_source):

  protected function query() {
    // Get the default query.
    $query = parent::query();

    // Add a join to content_field_video.
    $query->rightJoin('content_field_source', 'cfs', 'cfs.nid=nr.nid AND cfs.vid=nr.vid');
    // Add the condition for the eyeblast provider.
    $query->fields('cfs', array('field_source_nid'));

    return $query;
  }

along with a prepareRow() method to put the data in the expected variable name:

  function prepareRow($row) {
    $row->field_source = $row->field_source_nid;
  }

Although I suppose another way would be to define field_source_nid as a custom source field and then the prepareRow() stuff wouldn't be necessary, but it seems like six of one and half dozen of another to me.

So for migrate_d2d gurus, am I reading this correctly? Is this a bug, or is there an existing way to get data from commonly shared fields in D6 and older that I'm missing?

Thanks.

Comments

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Joining directly to the content_field_* tables is not done, because another reason those tables might exist is for fields with multiple values - if you join to the field table and a node has multiple values for the field, this would cause multiple rows for the same node in the migration process. For this reason, data from the content_field_* tables is added in DrupalVersion6::getSourceValues() (look at line 271 forward). That's the area to debug to understand why you're not getting your CCK field values. FWIW, I just tested with a D6->D7 migration of my blog and an image field which lives in content_field_images migrated fine.

wonder95’s picture

Status: Postponed (maintainer needs more info) » Closed (works as designed)

Sigh. Yes, I see the code, and it's working now. I must have had something screwy in my code. For some reason, DrupalVersion6::getSourceValues() wasn't even being called, but no problem now.

fonant’s picture

@wonder95, can you say what it was that was screwy in your code?

I'm having exactly then same problem: Migrate is happy with the field mapping, but doesn't get the field data.

wonder95’s picture

@fonant sorry, it's been a while, and that was a pretty large migration, so unfortunately I don't remember exactly what I fixed. If I do remember, though, I'll post it back here.

fonant’s picture

Found it: In the sub-class prepareRow() method you have to call parent::prepareRow(), like this:

  public function prepareRow($row) {
    if (parent::prepareRow($row) === FALSE) {
      return FALSE;
    }
  ...