Migration did not meet the requirements when using cache.backend.null

Replicated this issue. With cache.backend.null enabled in settings.php, required migration dependencies are not discovered. Disabling cache.backend.null allows the dependent migrations to be discovered.

Not sure if this is a bug or a feature....in any event it should be documented somewhere.

Can reproduce the issue when the cache.backend.null enabled in development.services.yml as well .

services:
  cache.backend.null:
    class: Drupal\Core\Cache\NullBackendFactory

#2797505: Migrations fail due to missing dependency when dependency has skipped rows by the source plugin
This might be related?

We're running into this issue with 9.3.x now... still very problematic!

Patch #61 from #2797505: Migrations fail due to missing dependency when dependency has skipped rows by the source plugin does NOT fix it for us.

I am having trouble reproducing this. I wonder if the situation has been improved at all after #3190815: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O)?

It looks to me like the error should happen if 1) there is no caching and 2) if the required migration's source plugin's count() function does not return the same number of items that were migrated. I think the issue I referenced cleaned this up for the core source plugins.

It would be great to get more details on how to reproduce. What is the exact version of D9? What is the definition of the migration that is failing? What are the definitions of the dependencies that are causing it to fail?

I think @danflanagan8 is right, we need clear steps on how to reproduce the problem. I also tried to reproduce the problem and found that the dependencies were always built correctly.

Adding tag for an Issue Summary update for adding the Steps to Reproduce. Since work can't progress without that changing the status as well.

Thanks, @danflanagan8 and @quitone for your replies!

I will create a clean Drupal 9 instance for Migration and document each step of our setup / migration process here, when I get the time.
(Hopefully before the upcoming Holidays)

Ok, I am sorry as @danflanagan8 pointed out, the reason, why our migration fails is that the count() function does not return the same number of items that were migrated.

Migration upgrade_d7_field_instance_label_description_translation does not meet the requirements. Missing migrations upgrade_d7_field_instance. requirements: upgrade_d7_field_instance. 
[error]  upgrade_d7_field_instance_label_description_translation migration failed.

This is because we are skipping a couple of entity_types we don't want to migrate, using "hook_migrate_prepare_row". Still wondering why the patch from #2797505: Migrations fail due to missing dependency when dependency has skipped rows by the source plugin does not work for us, but this is a problem for another thread.

@Grevil, thanks, that clears this up a lot.

Although rows can be skipped in prepare_row it also means that the row is not saved in the map table as ignored. This will cause unexpected issues with counts. If you can modify your migration so that the rows are skipped in the process pipeline you should see an improvement. For example, what I have done in the past in prepare_row is to identity the row that needs to be skipped and set a property on the row to signal the skip to FALSE and then in the pipeline do this.

process:
  skip: 
    plugin: skip_on_empty
    method: row
    source: skip_this_row

@Grevil, any luck with moving the skip to the process pipeline?

@quietone, we currently postponed the project, but I will let you know in a couple of weeks, when we are at it again! :)

@Grevil, I hope the work on the migration project resumed and went well for you.

The original problem in the IS was researched and tested by @danflanagan8 in #12 and was found that it could not be reproduced. And in #15 @Grevil confirmed that. Following that I made a suggestion to fix the mis-matched counts, which is really a support conversation not a bug. Therefor, I am closing this issue as outdated.

I've updated the IS with the findings.

Sorry for the late feedback @quietone! The problem was a duplicate database entry in the source database...
For more information, visit: https://www.drupal.org/project/drupal/issues/3307506

But you already commented that issue anyway. Still thanks everyone!

I've found that this issue seems to occur when I use drush to run a migration - drush migrate:import users_migration, which has a required dependency on users_roles_migration, results in this error:

Migration users_migration did not meet the requirements. Missing migrations users_roles_migration. requirements: users_roles_migration.

However, if I run the migration through the Migrate Tools UI it executes correctly and runs the required dependencies first. I've got $settings['cache']['default'] = 'cache.backend.null'; commented out.