In some scenarios there are some facts about the data source, that I want to be sure are true.

For example:

  • Textlength of a field < x
  • Integerfield in a specific range
  • Other, more complex assertions

a) Those things could be checked during an import. How to do error reporting in prepare or prepareRow?

b) A seperate method, just for assertions and something like drush migrate-check Migration would be a useful feature.

Comments

mikeryan’s picture

Assigned: Unassigned » mikeryan

You got me thinking about a related subject: #1152878: Add source analysis. I started down that line here, but I think that is a different solution than the assertion case.

For assertions, I think it makes most sense to add them to field mappings, which also helps make the expectations visible (we can display the assertions on the migration info pages):

$this->addFieldMapping('field_count_of_something', 'source_count')
     ->assertIntegerRange(0, 10);
$this->addFieldMapping('field_signature', 'source_signature')
     ->assertStringLength(0, 255);

Assertions could be tested in applyMappings, and assertion failures added to the message table.

Thanks!

mikeryan’s picture

Title: Feature: Define assertions on the data source » Define assertions on the data source
Niklas Fiekas’s picture

Looks good. How would I add "custom" assertions there?

mikeryan’s picture

Perhaps

$this->addFieldMapping('field_complex', 'source_complex')
     ->assertMethod(t('Validate the value is < 28 but not 17'), 'myAssertMethod');
...
protected function myAssertMethod($row) {
  $complex = $row->source_complex;
  if (!($complex < 28 && $complex <> 17)) {
    $this->saveMessage(t('Complex value !value is out of range', array('!value'=>$complex));
  }
}

Of course, you can always do what you want in prepareRow().

Niklas Fiekas’s picture

Awesome.

Some details:

  • If an assertion fails: Continue anyway / skip row and continue with the next one / abort import?
  • Swap parameters of assertMethod, make description optinal?
mikeryan’s picture

If an assertion fails: Continue anyway / skip row and continue with the next one / abort import?

You can do the first two today in prepareRow - do nothing beyond saveMessage() to continue, return FALSE to skip the row. You could also abort the import by setting the status column in migrate_status to MigrationBase::STATUS_STOPPING.

Don't expect this to go into Migrate soon - I'm going to be releasing a beta of Migrate 2.1 shortly, I'll save this for Migrate 2.2. For now, I would recommend just doing your assertions in prepareRow().

Niklas Fiekas’s picture

Don't expect this to go into Migrate soon - I'm going to be releasing a beta of Migrate 2.1 shortly, I'll save this for Migrate 2.2. For now, I would recommend just doing your assertions in prepareRow().

Yes. I don't really need assertions for my current project. Only I found at some points, that they would be handy.

joachim’s picture

> Other, more complex assertions

Would getting something like the MigrateTaxonomyTermReferenceFieldHandler to file a log message if it fails to get a taxonomy term from the incoming data come under this heading? Not sure whether to file a new feature request or pile in on this one :)

mikeryan’s picture

That falls under the heading of runtime notice (doing a saveMessage with MigrationBase::MESSAGE_INFORMATIONAL), it's not an assertion on the source data.

pifagor’s picture

Issue summary: View changes
Status: Active » Closed (outdated)