The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.Problem/Motivation
Marking a row needs-update has no effect if mapJoinable() = FALSE and Highwater Marks is ON.
To reproduce:
1. Create a migration with highwater marks ON and a Drupal\migrate\Plugin\migrate\source\SqlBase source that is not mapJoinable (eg. a D7 database).
2. Run the migration to completion.
3. Mark one or more records in the map table as needs-update (source_row_status=1).
4. Run the migration again and watch as zero records are updated.
Proposed resolution
Modify Drupal\migrate\Plugin\migrate\source\SqlBase::initializeIterator() such that if any records in the map table are marked NEEDS_UPDATE, highwater marks optimization is not added to the query.
As an example, the same fix was applied to D7's migrate module:
Issue: #2379289: migrate-import --update does not seem to work as expected, if map is not joinable, due to highwater field?
Commit: https://cgit.drupalcode.org/migrate/commit/?id=6f2981b
Remaining tasks
Patch, test(s), review.
Comments
Comment #2
sylvainm commentedI wrote this patch, inspired by the one in d7 migrate.
Tomorrow, i'll try to add a test
Comment #3
sylvainm commentedForgotten the file…
Comment #4
DeFr commentedThis should use getRowsNeedingUpdate() on the id map instead of doing a query directly.
Comment #5
sylvainm commentedThx
Comment #6
sylvainm commentedWell, here is a test.
Comment #9
DeFr commentedI think it'd be nice to have a test that only sets some of the node as needing updates instead of all of them, and checking that only those that should be updated are actually updated even though the high water condition isn't added to the query.
Namely, it'd be good to have:
Comment #12
quietone commentedClosing this as a duplicate of #2859314: Highwater condition with unjoined maps skips unprocessed and NEEDS_UPDATE rows. I'll move credit over there.