Migrate Dump export needs to have a consistent order

We need to update migrate-dump-d6.sh to add an order by primary key to the exported data. I think we should be able to calculate the primary keys automatically from the schema data we already have.

After we've committed the updated table files everyone will have the same results while exporting the Table's in the future.

I've been working on this, enjoying it too, and am about half way done. I hope it is OK that I've assigned it to myself.

@quietone, what's your approach? I was hoping to calculate the primary keys automatically based on the available schema data?

@benjy, I was working off your suggestion in #4 and used this to get the primary keys.

 $query = "SELECT `COLUMN_NAME` FROM `information_schema`.`COLUMNS`
  WHERE (`TABLE_SCHEMA` = 'd6_migrate') AND (`TABLE_NAME` = '{$table}') AND (`COLUMN_KEY` = 'PRI')
  ORDER BY COLUMN_NAME";

That seams reasonable, the only thing I did think was that the data may already exist in $row ?:

  while(($row = $query->fetchAssoc()) !== FALSE) {
    $field_name = $row['Field'];

Yes, but I don't see how that helps. Using 'key' from a row as you suggest would mean that the code would have to do extra sorting. Why not let the db do that? Or, I'm just plain missing something?

Note that my patch only changes the $values output, which is where the differences are in d6_dump_no_changes.diff. I presume $fields should be as well?

Took another look, I see what you mean. Yes, I like this approach.

Did we have a patch with the newly generated dumps? Hopefully it's not too much of a radical change.

Yes, here it is. 12 dump files changed.

I'm getting AJAX errors when trying to run the tests, so I'll see what the testbot finds while I investigate.

Right, increased the PHP memory limit and tests pass locally. Is the testbot sick today?

Honestly, suggestions?

Yes, I think testbot is just having issues.

test again

Try again

And again

Sure. Here it is.

Sorry that's still not a binary dump :/ but this is :) Otherwise no change.

To roll this, you need core/modules/migrate_drupal/src/Tests/.gitattributes to contain *.gz -diff.

@chx, thanks. As they say, you learn something new every day. But I think you meant *.gz -binary?

Yes, I work from HEAD.

core/modules/migrate_drupal/src/Tests/.gitattributes does not exist, nor do I see one on GitHub.

Yes, *.gz -text -diff works with git diff --full-index --binary. Seems like I'm a lazy typist and was using "git diff" and thus needed different .gitattributes file to get a patch that matched chx's.

Needs a reroll

Done

Let's try again.

Migrate is not frozen in beta. Committed e975bb1 and pushed to 8.0.x. Thanks!