system_post_update_sort_all_config can exhaust PHP memory in 9.3.0 because processing is not batched

Thanks for working on this.

1. +++ b/core/modules/system/system.post_update.php
   @@ -208,9 +208,29 @@ function system_post_update_delete_authorize_settings() {
   +  $iteration_size = 50;
   

   I think this should use entity_update_batch_size:

   /**
    * The default number of entities to update in a batch process.
    *
    * This is used by update and post-update functions that need to go through and
    * change all the entities on a site, so it is useful to increase this number
    * if your hosting configuration (i.e. RAM allocation, CPU speed) allows for a
    * larger number of entities to be processed in a single batch run.
    */
   $settings['entity_update_batch_size'] = 50;
   

2. +++ b/core/modules/system/system.post_update.php
   @@ -208,9 +208,29 @@ function system_post_update_delete_authorize_settings() {
   +  if ($sandbox['max'] > 0) {
   

   Not sure this is necessary, there must be at least one config entity?

Fixed CS error.

Looks good, ran it locally and get a nice progress bar and everything.

Took a look at maybe simplifying the logic a little bit to make it a little easier to read, but this works so ¯\_(ツ)_/¯

Over in #3254169-34: Update aborted by system_post_update_sort_all_config. Drush command terminated abnormally. Output is empty. @bmango reports:

I have applied the patch from #3254403: system_post_update_sort_all_config can exhaust PHP memory in 9.3.0, and I am still getting out of memory. I also flushed caches.

Posting just to add I don't think this should stop this being committed, as batching the update can only make it better - even if it hasn't solved it for everyone.

Would normally use array_chunk() for an update function like this, but the for loop is fine.

If this was all config entities we could have used config entity updater, but it's not, so the manual batch is OK. And even though they're not entities, using the entity_update_batch size seems correct here.

Committed/pushed to 10.0.x and cherry-picked to 9.4.x and 9.3.x, thanks!

Have applied the patch and still getting out of memory.

Are you certain the system_post_update_sort_all_config function is executing when that occurs?

Yes, I am 100% sure

It worked on sanitized database, but not on prod. Set php and drush memory limit to 2G. Still not working. Tried even set to 3G, but drush limit not taking 3G. We are using mariadb 10.5.12

What is entity_update_batch_size set to on that instance?

in settings.php: $settings['entity_update_batch_size'] = 50;

Let’s leave this one fixed and work the other issue for whatever cases don’t seem to be fixed by batching.

I did upgrade my Drupal from 8.9.x to 9.3.2 and I am still facing the issue to perform database updates. In 9.3.2 no need to apply the patch however going through this thread I have reduced $iteration_size = 1; and still memory utilization going more than 100GB and after some time it throws "The process has been signaled with signal "9". "

Please find the attached screen for the same. Does anyone have any solution to this?

This issue is resolved. Please see #3254169: Update aborted by system_post_update_sort_all_config. Drush command terminated abnormally. Output is empty.

Run first, drush cim before updatedb