Problem/Motivation

Provide quasi-patch package on ftp.drupal.org of updates from one point release to another. See #3055872: In place updates (from quasi-patch zip), which has the format for the quasi-patch.

Proposed resolution

Remaining tasks

  • ✅ Create a DB table to track quasi-patch status, with columns from_release_nid, to_release_nid, time requested, time generated, error if not generated, intermediate key expiration.
  • ✅ Add in-place-updates menu callback as a fallback for files that have not been generated. It will validate the request, return a “try again soon” response, add a row to the quasi-patch status table, add to a queue.
  • ✅ Resolve https://github.com/drupal/php-signify/issues/13, so intermediate key expiration can be stored.
  • ✅ Set up a queue runner to generate the quasi-patches.
  • ✅ Set up cron to remove files nearing their intermediate key expiration date.
  • ✅ Add integrity checking to monitor for errors.

User interface changes

API changes

Data model changes

Release notes snippet

Comments

heddn created an issue. See original summary.

drumm’s picture

Assigned: Unassigned » drumm

To start, I think I’ll only generate these for sequential patch versions. If someone skips from 1.2.3 to 1.2.5, nothing is stopping them from going back and releasing 1.2.4, for example.

  • drumm committed 1450ec4 on 7.x-3.x
    Issue #3061248: Also refactor out function for only signing data
    
  • drumm committed 1d3620c on 7.x-3.x
    Issue #3061248: Refactor out useful functions
    
  • drumm committed be72ff9 on 7.x-3.x
    Issue #3061248: Add doc comment
    
  • drumm committed f95e827 on 7.x-3.x
    Issue #3061248: Package quasi-patch for automatic updates
    
drumm’s picture

These updates are now available as examples:

They include a checksumlist.csig file signed with the temporary keys.

The autoupdates client should plan on requests potentially failing, ideally retrying maybe 3 or 4 times, with maybe 20 seconds between requests. I realized that dong the version number math of what the “incremental” change would be would be hard to get right, with plenty of edge cases. As this moves closer to production, I plan to have the initial request go to a queue, and respond immediately that it is being worked on. Once the processing for the item completes, it is just a static file that will be served quickly and cached.

drumm’s picture

Issue summary: View changes

Adding notes on queue architecture to the issue summary.

drumm’s picture

Issue summary: View changes

  • drumm committed 78ee4b4 on 7.x-3.x
    Issue #3061248: Write zip contents to the top level of the archive
    
drumm’s picture

The files in #4 have all been regenerated with the top-level directory in the .zip files removed for easier processing by the client.

  • drumm committed b1d6256 on 7.x-3.x, dev
    Issue #3061248: Also check that signed data matches original data
    

  • drumm committed f9f958a on 7.x-3.x, dev
    Issue #3061248: Update the php-signify library for https://github.com/...

  • drumm committed 8b612c3 on 7.x-3.x, dev
    Issue #3061248: Add tracking for requested in place updates
    
drumm’s picture

Issue summary: View changes

The menu callback to handle in-place-updates that have not been generated yet, and associated tracking, is now deployed.

Requesting arbitrary update combinations like https://www.drupal.org/in-place-updates/drupal/drupal-8.7.4-to-8.7.7.zip will now be recorded, and respond with an HTTP 429 and a Retry-After: 10 header.

Currently queue processing is not set up, so these will not be generated until I run a drush command.

  • drumm committed 4045c76 on 7.x-3.x, dev
    Issue #3061248: Add queue for in-place-update generation
    
drumm’s picture

Issue summary: View changes
Status: Active » Needs review

Queue processing for this is now all hooked up. Arbitrary requests to this API will serve in a production capacity, except for the signing being with test keys until #3076191: Build a signing oracle server & HSM infrastructure is done.

heddn’s picture

NICE!!!

  • drumm committed a2827eb on 7.x-3.x, dev
    Issue #3061248: Add integrity checks
    
drumm’s picture

Issue summary: View changes

Integrity checks are now in, just need clearing expired files now.

  • drumm committed 67e9414 on 7.x-3.x, dev
    Issue #3061248: Clear expiring in place updates
    
drumm’s picture

Issue summary: View changes
Status: Needs review » Fixed

Files that are expiring will now be deleted around 48h before they expire.

I think this resolves everything we need to resolve in this issue. The updates are still signed with a temporary key, until #3076191: Build a signing oracle server & HSM infrastructure is done.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.