Problem/Motivation

Consider a SQL-based migration using highwater marks:

source:
  plugin: my_node
  node_type: my_article
  high_water_property:
    name: changed

On initial import, everything works fine, and a highwater mark is saved. Then you do an incremental migration, and because that highwater mark exists SqlBase does this:

      if ($this->getHighWaterProperty() && ($high_water = $this->getHighWater()) !== '') {
        $high_water_field = $this->getHighWaterField();
        $conditions->condition($high_water_field, $high_water, '>');

So, your query has "WHERE changed > 123456789" or somesuch in it.

Then, you rollback the migration, and SourcePluginBase does this:

  public function postRollback(MigrateRollbackEvent $event) {
    // Reset the high-water mark.
    $this->saveHighWater(NULL);
  }

When you next try to import, that SqlBase bit adds "WHERE changed > NULL" and you get no rows. Boo!

For now, here's a quick-and-dirty work-around when you get stuck in this situation:

update key_value
set value=''
where collection='migrate:high_water' and value='N;'

Proposed resolution

Ideally, I'd like to change SqlBase to check the highwater mark for general FALSiness rather than explicitly against '' - but, I have a vague recollection of our doing this for a specific reason. I can't seem to find the issue where this was discussed, though.

Alternatively, we could have postRollback() save ''.

Remaining tasks

  1. Write a test demonstrating the failure (setup a highwater migration, execute it, roll it back, then execute it again and make sure it actually imports stuff).
  2. Implement one of the two solutions above (novice).

User interface changes

N/A

API changes

None

Data model changes

None

Comments

mikeryan created an issue. See original summary.

mikeryan’s picture

Issue summary: View changes
mikeryan’s picture

Issue tags: +Novice

The actual fix should be novice - testing it is, I think, semi-novice...

mikeryan’s picture

Status: Active » Closed (duplicate)
Related issues: +#2865497: Fix high-water condition for new migrations

Oops, there's an already an issue to address this: #2865497: Fix high-water condition for new migrations .