Handle various migration interruption scenarios

As I prepare to hit Save, it occurs to me that some mocking in existing tests may be necessary... Well, let's see...

This has been verified to work for the --limit scenario with the patch at #2432983: Implement --limit option on migrate-import.

Hit Save a little quick there... This version adds the ability to clear the interruption (kinda important so a given migration doesn't get permanently interrupted), and a test.

This is blocking a few tools features.

This blocks a few features for the Drush commands in progress.

Maybe the interruption stuff could be moved into $this->checkStatus() ?

The interruption stuff is replacing checkStatus, which was historically a grab-bag of various reasons to interrupt a migration. The current patch here was rolled between removing the memory checking from core, and re-adding it, so for a brief shining moment checkStatus() did not exist. The follow-up in #2545632: [PP1] Move memory reclamation out of migrate executable that separates out the memory checking to an event handler should use the interruption stuff.

In general I think import() needs some refactoring, the method is becoming a little complex to follow.

Any specific thoughts? One particular thing I'd like to revisit (someday) is the queued messages business, a holdover from D7...

Rerolled, with a comment on the hopefully soon-to-be-doomed checkStatus()...

We already have both getMigrationResult() and setMigrationResult() which use the same RESULT_* constants. Can we use the same methods and check in migrate executable for some kind of interruption result?

The original approach was to set RESULT_STOPPED as the interruption result. However, as detailed in the IS, we want import() to return different results in different scenarios. setMigrationResult() doesn't allow us to both signal the interruption and to provide the result to use upon interruption.

Or better yet, we could start using the STATUS_* constants that we already have and check the migration status in the loop and save result for the completed statuses only?

The STATUS_* constants represent the disposition of a given source row, to be saved in the map table. The RESULT_* constants represent the final status of an import process, to be reported by the front-end. Apples & oranges...

Extra white space

Fixed.

The STATUS_* constants represent the disposition of a given source row, to be saved in the map table.

Oops, wait, I was thinking of the MigrateIdMapInterface::STATUS_ constants, you meant the MigrationInterface constants, which will get used when I get back to #2429085: Track current state of migrations. So, we could signal interruption by setting MigrationInterface::STATUS_STOPPING in state, but we still also need to be able to specify the RESULT_* to be returned by import().

MigrateIdMapInterface: these statuses represent the disposition of a given *source* row (*map* row below is confusing - although the status is stored in a map row it's describing what happened the last time the source row was processed). This is completely orthogonal to the status of a migration.

  /**
   * Codes reflecting the current status of a map row.
   */
  const STATUS_IMPORTED = 0;
  const STATUS_NEEDS_UPDATE = 1;
  const STATUS_IGNORED = 2;
  const STATUS_FAILED = 3;

MigrationInterface: these statuses represent the status of a migration process at this moment - what is it doing (if anything)? They are primarily for use by #2429085: Track current state of migrations - they enabled migration tools (drush migrate-status command, dashboard UI) to report what a migration is doing now - as well as providing a semaphore so, say, one person doesn't start a rollback operation on a migration while someone else is running an import). This is completely orthogonal to the status of any given source row handled by the migration.

  /**
   * The migration is currently not running.
   */
  const STATUS_IDLE = 0;

  /**
   * The migration is currently importing.
   */
  const STATUS_IMPORTING = 1;

  /**
   * The migration is currently being rolled back.
   */
  const STATUS_ROLLING_BACK = 2;

  /**
   * The migration is being stopped.
   */
  const STATUS_STOPPING = 3;

  /**
   * The migration has been disabled.
   */
  const STATUS_DISABLED = 4;

MigrationInterface: these result codes represent what happened with the migration process that has just completed, so the front-end runner can report what happened to the user. This has some relationship to the MigrationInterface status constants (e.g., a status of DISABLED leads to a result of DISABLED), but it's not a 1-to-1 relationship (as explained in the IS, we may want to interrupt the migration and have a result of COMPLETED).

  /**
   * All records have been processed.
   */
  const RESULT_COMPLETED = 1;

  /**
   * The process has stopped itself (e.g., the memory limit is approaching).
   */
  const RESULT_INCOMPLETE = 2;

  /**
   * The process was stopped externally (e.g., via drush migrate-stop).
   */
  const RESULT_STOPPED = 3;

  /**
   * The process had a fatal error.
   */
  const RESULT_FAILED = 4;

  /**
   * Dependencies are unfulfilled - skip the process.
   */
  const RESULT_SKIPPED = 5;

  /**
   * This migration is disabled, skipping.
   */
  const RESULT_DISABLED = 6;

So, I think what people are asking for is rather than a specific API for interruption, the client doing the interrupting should do

$migration->setStatus(MigrationInterface::STATUS_STOPPING);
$migration->setMigrationResult(MigrationInterface::RESULT_INCOMPLETE); // Or whatever result

I still like the idea of a specific API, though, that does the above - rather than setInterruptionResult($result), perhaps interruptMigration($result) would be clearer (as opposed to some obscure d.o doc page describing how to interrupt a migration and set the result).

But, now I'm looking at setMigrationResult() and getMigrationResult() and getting uncomfortable. The RESULT constants are meant for communicating the immediate result of a migration result to a front-end client - why is it being saved in the keyvalue store? It turns out getMigrationResult() is used in exactly one place - isComplete() - which is in turn used in exactly one place to validate migration_dependencies (i.e., that the dependent migrations have run to completion). This is, unfortunately, wrong - the COMPLETED result is returned to the frontend to indicate that the requested migration *operation* has completed, and if it was requested to be a partial migration (--limit, --idlist) COMPLETED is returned, although the complete migration has not been run. So, that's a problem - out-of-scope for this issue though. It is related - fixing that issue on its own would remove the need for the result to be put in the kv store for that purpose, but the approach above would still need that...

isComplete() issue at #2554003: isComplete() should not rely on RESULT_COMPLETED.

If we're going to use setStatus(), #2429085: Track current state of migrations needs to get in first.

Unblocked, will reroll the patch to use getStatus.

Here we are, leveraging the status.

With a little extra test suggested by phenaproxima.