In #2704799: RFC: Add a 'distribution' content type I implemented a simple 'distribution' content type, along with 'upgrade' tasks for sites and platforms. It was intended initially to simplify the site-owner experience of keeping their site up-to-date. However, distributions should also form the basis of a number of other platform maintenance and automation improvements.

Ideally these should be focused on specific workflows and capabilities, rather than lower-level concepts and functionality. For example:

  1. Track a drupal.org distribution's releases: We could add a field for the name of a distribution on drupal.org, then implement a queue that checks the latest release. Whenever a new release is detected, the queue would trigger a new platform build (pointing to our distribution), then set the upgrade target to this latest platform, perhaps also locking all other platforms in the distribution. The first step (the platform build) might be a bit complicated, as we might have to download and untar the distro in a pre-verify hook, or perhaps use drush make with a temporary makefile where we specify the distro as a "core".
  2. Track a Drush Makefile: We could add a makefile (or lockfile) field, and store a hash of it. That should allow a queue to detect changes, then trigger the rest of the process (platform build, update to upgrade target field, etc.)
  3. Track a platform git repo: Similar to the above, but we store and compare the latest commit hash on a given branch, perhaps, or look for new tags.
  4. Track a composer-based platform: Again, similar to the above, but supporting the platform living in a sub-directory, and running "composer install" after cloning a git repo. This would assume a git repo initialized by something like this.

I currently work mostly with a #2-style workflow, but my recent D8 projects are definitely more along the lines of #4.

Comments

ergonlogic created an issue.

ergonlogic’s picture

On further reflection, I think #3 and #4 could be collapsed, with a couple checkboxes to enable the optional stuff in #4. The difference is really only in whether vendor/ was committed into git (as in #3) or not (as in #4).

The git-based platform build should perhaps be handled in a pre-platform-verify hook (or similar), as opposed to using the hosting_git platform functionality. My reasoning here is that these are intended to be stable, immutable production platforms, and that their usage ought to follow Aegir's best practices that allow automatic rollbacks, etc. So, once built, we shouldn't be running any further git or composer commands on these platform, and so we shouldn't require the overhead of storing said data in the platform context/alias.

Speaking of Aegir contexts, so far I've intentionally avoided any back-end functionality specific to distributions. That is, we don't currently create a @distribution_example alias/context, as we do for sites, platforms and servers. We could do so, which would then allow us to add a provision-upgrade command.

However, consider that provision-migrate only runs on site contexts in the back-end, whereas platform-level migrations can only be triggered from the front-end. This is because we don't assume access to the database when operating on the back-end, and so we don't have an easy, reliable way to determine which sites live on which platforms. The same limitation would exist for distributions on the back-end. That is, every time we'd add a platform to a distribution, we'd need to re-write the distribution context as well, so as to maintain a list of platforms. But to allow for a platform provision-upgrade command, we'd also need to add a list of sites to platform contexts.

Anyway, my point is that seeking to support headless operations for distributions would likely be messy and involve more work than is justified.

ergonlogic’s picture

I have a working protoype of Drush Makefile tracking (#2 in the issue summary). That is, if you update a makefile (on github, for example), new platforms are built automatically. Every relevant platform and site will then have a simple "upgrade" task button appear that will handle the update without further contextual knowledge required.

tvl’s picture

Sounds really cool! How can we test it?

ergonlogic’s picture

I'm re-factoring both distributions and the update checks to be more object-oriented. Once I've done that, I'll probably split off the distributions stuff into a separate module (it's currently under http://drupal.org/project/hosting_dev). Feel free to test it, but understand that it's under heavy development at the moment, and probably won't stabilise for another week or two.

ergonlogic’s picture

As I mentioned in #2704799: RFC: Add a 'distribution' content type, I've split Distributions into a separate module: https://www.drupal.org/project/hosting_distribution. It appears to be reasonably stable, and includes #2 above (track a Drush makefile via a queue). I'll release a beta shortly, to encourage testing.

#4 is largely dependent on #2838489: Support drupal being in a subdirectory of a git repo.

ergonlogic’s picture

I just released a second beta, that adds support for distributions based off of drupal.org-hosted stock distributions. That is #1 above.

ergonlogic’s picture

I've just added support for a Git update plugin, along with some follow-up issues.

ergonlogic’s picture

With the latest commit, I added (perhaps prematurely) support for the solution I posted to #2838489: Support drupal being in a subdirectory of a git repo.

This, combined with the patch in https://www.drupal.org/node/2736801#comment-12034550 makes for a reasonable git/composer workflow for platform management.