Structure:

  • Drupal 8 latest, Drush 9
  • Local => Dev => Prod structure
  • Code managed by composer, git, drush
  • Main Dev repo on bitbucket
  • Custom profile

Workflow

  • We pull the Main Dev Code
  • We do development on feature branches on locals and push to main repo
  • After review, we merge
  • On the Main Dev Server, we fetch, rebase, import configs(drush cim)

Problem
This setup was working perfectly until we noticed the following:

  • After attempting to export config (drush cex), we noticed that drush was proposing to delete all the configs in our profile/custom_profile/config/sync; surprising because normally it will create new ones, update modified, and delete removed ones.
  • If we approved the deletion, drush will proceed and says all configs where deleted, but the profiles/custom_profile/config/sync still have all the configs, so nothing was deleted.
  • When attempting to import new configs, the system acts like it want to create all the configs, like if there is no existing config in the database
  • If we approved, we get the following errors:
    1. [warning] array_diff_key(): Argument #1 is not an array ConfigImporter.php:380
    2. Site UUID in source storage does not match the target storage.
    3. Cannot change the install profile from to custom_profile once Drupal is installed. in Drupal\Core\Config\ConfigImporter->validate() (line 737 of /home/devxx/custom_profile/web/core/lib/Drupal/Core/Config/ConfigImporter.php).
  • No config change is made

Action

  • The first action was to delete the newly created configuration and re-attempt config export and import, and everything worked.
  • But we need to create new configs; so we checked to make sure there is no mis-configuration, and recreated new config; the export and import still fails with the same error.
  • The Site itself works fine; but we cannot import new configs (Create field, content type ...)

More digging

  • We checked the config files in profiles/custom_profile/config/sync; there is about 1005 files
  • We've noticed that if we have more of that number of files imported, the next import will fail.
  • We've attempted to keep last created config that made the import fail, and removed some old fields that where imported when the import was working, to bring the number around 1005 files, and the export and import works.

Current Situation
We are looking in our commits to see if there is no issues with committed UUIDs.
But, IS IT POSSIBLE THAT THERE IS A LIMIT ON THE NUMBER OF CONFIG FILES IN THE DATABASE?
I know this sounds crazy, but I have to ask!

Comments

sibopa created an issue. See original summary.

sibopa’s picture

Issue summary: View changes
cilefen’s picture

Priority: Critical » Major
Issue tags: -Configuration import; Drush config import

I am sorry to hear you are having issues with configuration. Does the problem occur only when using Drush? Can it be reproduced with a UI export and import?

sibopa’s picture

Thanks for your quick reaction.
Yes, the issue also happens when importing from UI.

idebr’s picture

Since you are hitting such a specific number of items, perhaps you are hitting php's configured max_input_vars?

sibopa’s picture

I did up the max_input_vars without success. And the problem is actually not there. As soon as the system checks the file and prompt about actions to be taken, you can see the discrepancy; instead of prompting to create new, update modified and remove deleted configs, the system prompts to re-create all the configs, like if there is no config in the database.

cilefen’s picture

sibopa’s picture

@cilefen Thanks for looking into this; I've seen that post already; but my scenario is different;
In my case, if I delete configs, regardless of when they where created and the entity they are from, until I reach the magic count of around 1000, the error goes away. And I can now cim and cex without problem.
So why is removing unrelated configs fix that site UUid issue all of the sudden?

sibopa’s picture

After digging, this looks like an ORACLE issue; I forgot to mention that my site was running oracle database; I stumbled on this oracle: ORA-01795: maximum number of expressions in a list is 1000 issue; and it was too close to the number where my site is breaking.
I spanned up a mysql database, installed my configuration in it, and it ran smooth, including import, export and everything else.
So Does anyone have experience with this oracle: ORA-01795: maximum number of expressions in a list is 1000 issue?
Is there a path for that?
Thanks!

kgoel’s picture

Project: Drupal core » Oracle Driver
Version: 8.8.x-dev » 8.x-1.x-dev
Component: configuration system » Code