If I understand correctly, it can currently happen that an event causes the container to be rebuilt and replaced in the middle of a request.
Services that originate from the old container may still be in use somewhere, whereas others are replaced.
And then there are the "persisted services"..

Somehow this all seems to work, but it is scary to think about.
I do not claim to perfectly understand all of it, not even close.
But I bet that a lot of things would be easier if the container would not change in the middle of something, but only at specific points in the request where it is "safe".

Speaking in a metaphor:
Don't tear down + rebuild a house from the inside. Do it from the outside.
And if you are still inside and think that you want to rebuild it, take notes instead. Draft a plan of the new house, then leave the old one, and then tear it down and rebuild it following the draft.

In Drupal, this would mean:

  1. Create a container instance, and provide it with a "queue object".
  2. Use the container services to handle the request.
    Whenever a component (e.g. a form callback) wants to install a module or change configuration, queue it up for later.
  3. Shut down the container, forget all its services.
  4. Execute the queued operations, outside of any container.
  5. If any of the operations would trigger events that need container services, e.g. hook_install() stuff, create a new container instance to do this.

This would all go nicely together with #2272129: [meta] Introduce a low-level DIC to organize low-level services, but it may also work standalone.

Comments

donquixote’s picture

Issue summary: View changes
donquixote’s picture

Btw, this would also allow us to stop worrying about the $allow_dumping parameter.

donquixote’s picture

znerol’s picture

I agree that rebuilding the container mid-request is very adventurous (though extremely rare under normal circumstances) and I also agree that we should get rid of persist services completely. At this point in the release cycle I'd rather prefer to introduce lower-risk changes like #2281903: Refactor DrupalKernel::initializeContainer in preparation of a replacement for persist services and #2282371: Document the fact that it is not safe to reuse services across container rebuilds instead.

donquixote’s picture

Title: [meta] Never replace the container while services are still in use. » [meta] Never replace the container while services are still in call stack.

though extremely rare under normal circumstances

Not really.
The call stack is still full of obsolete services, when e.g. a form submit handler enables a module.
Usually these don't do much harmful, but I would certainly say it's a source of unpredictability. And we already have too much of that in Drupal.

znerol’s picture

normal circumstances == production environment

donquixote’s picture

At this point in the release cycle

I personally don't care that much..
The bootstrap, module install, container management, all that is currently a mess of unpredictability, full of stuff that people are afraid to touch. We continuously lose time working around these problems, and it will follow us around during the D8 life cycle. It won't become easier with contrib in the mix.

Some cleanup needs to happen before a Drupal 8 stable, and I would prefer a solid solution with a real plan.

normal circumstances == production environment

Module install does happen on production too..
Sure, not on the average visitor request. But weird stuff happening on module install can have side effects beyond that request.

I agree it might be rare that we really see issues because of this. It is just that I have grown a sensibility to anything that lacks or hurts predictability.

Btw I don't see you on IRC..

donquixote’s picture

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Active » Postponed (maintainer needs more info)
Issue tags: +stale-issue-cleanup

Thank you for creating this issue to improve Drupal.

We are working to decide if this task is still relevant to a currently supported version of Drupal. There hasn't been any discussion here for over 8 years which suggests that this has either been implemented or is no longer relevant. Your thoughts on this will allow a decision to be made.

Since we need more information to move forward with this issue, the status is now Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.

Thanks!

smustgrave’s picture

Status: Postponed (maintainer needs more info) » Closed (outdated)

Since it's been 3+ months without a follow up going to close out. If still a valid issue please please re-open!

Thanks all!

donquixote’s picture

I think this is a dead end issue :)
There are always services in the call stack, and so far this seems to be ok.
The services from the old container create a new container, and that's how it works.