Currently, it is not possible to perform a Drupal core update in-browser.

One of the substantial technical roadblocks to achieving this, to highlight a point raised in older issues / feature requests, is the fact that any sort of in-browser updates are best done incrementally as a batch of smaller work units, to avoid potential timeout issues. The existing module update manager works this way, for example. But, if you use core's facilities such as its batch api, authentication, ftp file writer, etc. to update itself, you run the risk of trying to use those facilities while they are partially updated and in an inconsistent state, leaving a site in a permanently broken state.

I have created a solution to this by developing a separate set of facilities for updating code through a browser, totally independent of Drupal core during the time the updates are being applied, as demoed at MWDS 2018, or below:

https://youtu.be/4yhO442LyII Oh look, now it looks all drupally: https://youtu.be/r3H-KzfV5sk

Code behind aforementioned demo:

This work is getting to the point where it could be used fairly readily to fulfill an initial objective of the automatic updates initiative (https://www.drupal.org/project/ideas/issues/2940731) -- providing manually installed, but via the browser core security updates. Because the code doing the heavy lifting is independent of Drupal core, it is approximately equally applicable to Drupal 7 and 8.

To use this successfully with Drupal, there are a few important requirements that would need support / buy-in from key Drupal and Drupal infrastructure stakeholders. There will hopefully be a future targeted srpint in which those issues can be discussed in person as well, but my motivation for writing this issue is mostly to raise awareness now and in hopes it creates a venue for some initial discussions / documentation about that. And also @xjm asked me to write an issue for it :)

The biggest of these requirements is that the release artifact my code downloads and consumes is not the full .tar.gz of the release. The reason is that it's advantageous to do the minimum work necessary to transform release n-1 to n, instead of blowing away the whole old release n-1 and then writing out the whole new one. This dramatically reduces tne amount of work the updater must perform, makes the update process run much faster / reduces site downtime, and probably reduces the number of things that can go wrong during the update. A better way to do it is to precompute the differences between release n-1 and n once as part of the release process and capture those differences in a release artifact that Curator downloads and replays to transform an n-1 release to n. The work to identify the differences between two arbitrary directory trees and suck them into a .zip file in the format Curator is looking for is over here, see https://github.com/curator-wik/makeup. So running this to generate the update-centric release artifacts would just need to be added to the Drupal release scripts, and the resulting static files would need to be hosted somewhere, presumably on the same infrastructure that hosts the full release archives. (Note it wouldn't make much sense to publicize these files as another download option for those who choose to update the traditional way; they'd hang out on your CDN but not be linked from webpages.)
-> For simplicity of evaluation purposes I'll stick some of these up on the Internet somewhere for now, and point the updater at that location for now.

A hopefully more minor issue that would need some review is that since there's no dependency on Drupal core during the update, the front controller / "index.php" that is serving the batch runner API is not Drupal's index.php. In the current D7 module manifestation of this, it is an alternative .php file that ships in the module, and I would expect that this works for most people on D7 because the default .htaccess in D7 doesn't prevent you from running .php files other than Drupal's index.php. However, in D8, the default is that you cannot run php scripts directly out of contrib. The likely / obvious solution to this would be a patch to `authorize.php` that hands over control to the Curator phar instead of bootstrapping Drupal and doing its usual thing if some request property exists like presence of a cookie that looks like a Curator session cookie.

I'm also happy to answer any specific questions folks have about design, implementation, approach to security concerns, etc. Code reviews and comments are even better! ;)

Comments

mbaynton created an issue. See original summary.

mbaynton’s picture

Issue summary: View changes
mbaynton’s picture

Issue summary: View changes

Link to new demo vid. Now it looks just like a job being run in Drupal's batch api while the update is being applied, for optimal UX. That's not actually what's going on, at all...

mbaynton’s picture

Issue summary: View changes

As promised by labor day, here's the Drupal 7 module part of it, as a sandbox project: https://www.drupal.org/sandbox/mbaynton/2997009. Please try it out!

arlingtonvoicellc’s picture

mbaynton, you've done impressive work solving a problem that has plagued Drupal forever. Thank you for the time you've invested. I hope stakeholders are paying close attention. To have any chance at surviving as a serious CMS, this issue needs to be solved. I hope the community gets behind your initiative. As I've alluded in a comment on another thread, I am finished building any new websites in Drupal until this issue has been solved in core or with a stable module. Best of luck moving ahead on this project. I'll check back occasionally to see if there's been progress on its adoption.

mbaynton’s picture

I've just added a rollback capability to the sandbox project. Now, if an error occurs while it is updating a site to a newer version, it stops and performs the opposite of all filesystem changes it had performed up until that point. That way, if e.g. one file has weird permissions, the site won't end up in an inconsistent, partially-updated state.

I'll start focusing on Drupal 8 next. I'm also getting closer to being comfortable un-sandboxing this for D7, but since failure has the potential to take down the sites of users who may lack the skills to easily recover, I would still like to see these additional things in place first:

  • More testers! The underlying Curator engine has reasonable unit, functional, and integration tests, but it's just one hosting environment. The more places this has been run, the better I'll feel.
  • Better release artifact generation. I'm manually running the script that builds new release artifacts after a new core release happens. Ideally these would also be generated/hosted on DA infrastructure.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mbaynton’s picture

Issue summary: View changes

Update link from sandbox to official contrib project

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.