Migration fails when a dependent migration has skipped rows

There is an inconsistency in how row skipping is handled:

- If one of the hooks invoked in SourcePluginBase::prepareRow() throws a MigrateSkipRowException, then the row is saved to the map.
- If one of the process plugins throws a MigrateSkipRowException, then the row is saved to the map
- If SourcePluginBase::prepareRow() returns FALSE to skip a row, then the row is NOT saved to the map, and we get this bug.

As a workaround, you can do the logic for skipping the row in SourcePluginBase::prepareRow(), then set a boolean for whether to skip the row to an arbitrary field name:

      $row->setSourceProperty('skip', $whether_to_skip);

and then use the skip_on_empty process plugin with an empty destination in the migration's config:

  null:
    plugin: skip_on_empty
    method: row
    source: skip
    message: 'Skipping!'

I think there is another case to take into account. If the destination key is already used. For exemple I have a migration which import files. The destination key is the "src" XML tag. If there are duplicate "src" elements, those duplicates will be counted as "unprocessed". In this particular case there are no exceptions thrown and prepareRow() does not return FALSE.

I'm going to up this to a major: a migration which uses prepareRow() as documented to skip a row then can't have another migration depend on it.

The crux of the problem is that things that want to check if a migration is complete just call the query, which returns a count. They don't do processRow() on each row. Meanwhile, actually running a migration lets you skip a single row and then considers that row as not existing in the source.

There are *two* different ways to count a migration's source rows and they *do not necessarily agree*!!

For example:

My migration source runs correctly:

> [notice] Processed 1980 items (1980 created, 0 updated, 0 failed, 0 ignored) - done with 'ex_group_membership_media'

But the status report says otherwise:

> Default (default) ex_group_membership_media Idle 1994 1980 14 2021-02-03 14:30:33

That's because in 14 rows, prepareRow() in my source plugin returned FALSE to skip a row.

The fix here is that prepareRow() should behave the same way as the catching of exceptions from processRow(): mark the row in the map as ignored.

@mpp, thanks for the report.
@joachim, nice analysis.

The 'wrokaround' suggested in #4 is not merely a workaround, it is, in my opinion, the correct way to skip rows.

This is a duplicate of #2797505: Migrations fail due to missing dependency when dependency has skipped rows by the source plugin. Closing this and moving credit there