Problem/Motivation
Follow-up to #3291770: Inform the site admin if Composer Stager's committer failed, possibly leaving the site in a half-updated state see general problem in that issue
The solution in #3291770 requires an exception to actually be caught. in the rare case where there is fatal which happens in commit the exception does not come back should also assume the site is half updated. Right now the site would just say there is an existing update that you could delete. it may or may not show the site has been updated depending on which files were copied.
Proposed resolution
To handle this problem we should write a PACKAGE_MANAGER_FAILURE.json to the active directory and delete this file as soon as Composer Stager finishes the commit operation. We should always check for existence of this fail and tell the site admin to restore from backup. The marker file's existence should be mediated by a service.
Issue fork automatic_updates-3293417
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
tedbowI am not sure if this solution should be in package_manager or Automatic Updates
Right now if a Drupal core update failed in the middle of an apply this would be very bad and your site should be restored from a back up.
On the other hand depending on how Project Browser uses Package Manager they could just copy over only the new module that was required. They could be very strict about checking to see if no Composer dependencies and other projects were added or updated. in that case if they were just using the new module directory as active/stage targets then if it failed to apply the folder could just be deleted and the user could try again. Drupal core or other code would not be affected.
UPDATE:
I just realized that if we don't have the
PACKAGE_MANAGER_UPDATE_FAILURE.txtfile then even though Project Browser won't affect core or other modules if it is installingnew_modulesub-sequent web requests there would be now way to tell thatnew_modulehad failed to be fully copied in.So maybe
PACKAGE_MANAGER_UPDATE_FAILURE.txtshould be set by package_manager and it should be up to other modules to determine how to respond.Actually maybe we should have PACKAGE_MANAGER_OPERATION_FAILURE.json. this that file we could have info like
which module was doing the operation and the time it started, etc
Comment #5
yash.rode commentedComment #6
yash.rode commentedComment #7
yash.rode commentedComment #8
yash.rode commentedComment #9
tedbowCurrently this MR creates the file that we want but it does not actually prevent another operation from going forward.
There is no check to be sure that if this file exists then Package Manager should not allow any operations.
so either we have to make an event subscriber to listen to PreCreate and stop new operations if the file exists. We would probably also want to a AutoUpdates Readiness check to also check this file.
Also we should add a check to
package_manager_requirements()to check for this file on$phase === 'runtime'so this means we would need to hard code the name of this file multiple places. Currently it is just in Stage.php.
So lets create an event subscriber validator in package_manger that does 1.
deleteFailureMarker()we will call this from stage.php. This means we will have to add the new validator service as a dependency of the Stage class in its constructor.Then we need kernel test for this validator that would
ApplyFailedExceptionthus leaving our mark fail in placeThe only thing this proposed test makes wonder is if we should actually prevent Delete if the file is present, something to think about later.
We still have the problem I mentioned in #2 but we can wait on that.
Comment #10
yash.rode commentedComment #11
yash.rode commentedComment #12
tedbowNeeds work. Suggested in the MR to open up a 2nd merge request for a different approach. see https://git.drupalcode.org/project/automatic_updates/-/merge_requests/39...
Comment #14
yash.rode commentedComment #15
tedbow@yash.rode thanks for the making the new MR to move everything into stage.
We also need to stop
\Drupal\package_manager\Stage::createand\Drupal\package_manager\Stage::claimfrom working if there is a marker file. I think they can both call a new method. `ensureNoFailedApply()` or a better name. That method should throw an exception if a marker file exists.Then we would need test coverage for that.
Basically after a update failed to apply if you tried to claim the stage or create a new 1 you should get this exception.
It think we should also figure out if we can handle this in
\Drupal\automatic_updates\Form\UpdaterFormone way would be to call `ensureNoFailedApply()` and if there is an exception tell the user in the UI that there was a failed exception.Comment #16
yash.rode commentedComment #19
yash.rode commentedComment #20
yash.rode commentedComment #21
yash.rode commentedComment #22
traviscarden commented@tedbow I asked you one question in the MR about an exception message. I also added a
@todoto add exception-handling when trying to read and parse JSON from a found marker file--on the supposition that it could get corrupted. I'll leave it to you to decide whether we should create a follow-up for that.Comment #23
tedbowNeeds work for MR comments
@yash.rode sorry this is tricky issue but an important one. But I think we are pretty close
Comment #24
yash.rode commentedComment #25
traviscarden commentedComment #26
phenaproximaI'd like to discuss a few of the design decisions here with @tedbow before proceeding. Self-assigning to do that.
Comment #27
traviscarden commented🙂
Comment #29
phenaproximaI've opened MR 471 to illustrate how I think this should work. To me, the failure marker should behave more like a stateful service that can assert the presence or absence of a particular marker file, as well as know how to clear it when needed. I don't think that logic should be embedded directly in Stage.
Thoughts?
Comment #30
phenaproximaComment #32
phenaproximaComment #33
bnjmnmI went through the code with @phenaproxima and this looks like a sound approach that is implemented well.
This would have properly identified problems I ran into with the Project Browser integration, as I had requires failing due to filesystem config. Having the
PACKAGE_MANAGER_FAILURE.jsonavailable would have noticeably reduced troubleshooting time.The issue summary should get updated to reflect the solution in the MR before this gets committed, but it is RTBC IMO.
Comment #34
phenaproximaComment #35
phenaproximaCrediting @bnjmnm for review.
Comment #36
phenaproximaIS is accurate.
Comment #38
phenaproximaMerged into 8.x-2.x. Thanks to everyone who pushed on this for so long!
Comment #39
yash.rode commentedHi phenaproxima, Removing package_manager/tests/src/Kernel/MarkerFileValidatorTest.php was a mistake or intentional?
Comment #40
phenaproximaIt was intentional. I kept the same coverage, just moved it into a couple new methods of StageTest. It didn't make sense to have it in a class called MarkerFileValidatorTest, since that makes it sound like there's something called MarkerFileValidator, which there isn't. :)
Comment #41
yash.rode commentedDo we need this method?
Comment #42
phenaproximaIf I’m looking at the right thing, yes. That’s how we override the failure message to be specific to Automatic Updates.
Comment #44
tedbow