hestenet (he/him)	Tim from the Drupal Association, in Portland OR.
tedbow	Ted, from Acquia, interested
xjm	:wave: xjm, core release manager
eiriksm	Eirik, from Norway, interested
xjm	Is @heddn around? :slightly_smiling_face:
heddn	@xjm thanks for the reminder. I'm here too
dts	waves
dww	Derek, interested
greg.1.anderson	Greg, SF CA. I saw a survey, had to vote. :slightly_smiling_face:
JayKandari	Jaideep, from India… Understanding the current state of autoupdates.. :slightly_smiling_face:

xjm	Core issue organization and issue tag for issues
xjm	Also the meta really needs an update
xjm	And finally, I'm curious about the cron improvements Lucas mentioned which were off my radar, and whether we're targeting that for core too
xjm	I guess I'm a fount of questions today... do we need to require Composer 2 for phase 2/core?
heddn	Topic: Current status of phase 2

xjm	So one interesting comment from that thread was about not having auto-updates running in production. Which "professional" sites definitely wouldn't, but long-tail sites certainly would because half the point is stopping a zero-day overnight or such.
xjm	At some point down the road we might want to define best practices
dww	To me the other interesting point from that thread was that having this for contrib means sites are depending on the maintenance practices of dozens / hundreds of contrib maintainers, all of which are wildly varying in their thoroughness and care.Plus all the inter-dependency hell of new releases not necessarily compatible with each other.
hestenet (he/him)	The proposed A/B autoloader design could help mitigate both of those elements - but that is of course still in the hypothetical phase of design.
heddn	WP went w/ contrib first. Drupal went w/ Core first. Such an interesting divide.
xjm	They did? Hasn't WP core had auto-updates for years longer, vs. plugins just now?
xjm	Like I remember WP-guy at the summit talking about their work on it
dww	AFAIK, they’ve had update manager-esque “new version out, click here to install” for both core and contrib for a long time. But not automatic updates as such?
heddn	I could be mis-remembering myself. I just know the latest "auto update" focus was all on contrib.

xjm	https://www.drupal.org/project/issues/search?projects=&project_issue_fol... please put that tag on issues so we can find them
xjm	Also the important ones should be at least major priority and ideally would be listed in the IS of the initiative roadmap
xjm	All of the phase 1 stuff aside from the actual updater module was intended to be stuff we could add to core without friction, so ideally we'd start getting stuff like the pre-flight checks into core
xjm	(Or filing issues against core I mean)
xjm	I only came across a couple because I was triaging the 9.0.x queue to clear out things that should have been filed against 8.8.x, 8.9.x, or 9.1.x instead
heddn	@xjm I haven't opened any issues to bring in the readiness checks into core
xjm	@heddn When you have a chance, can you put the issue tag on any core issues you did file? I only found the three
heddn	@xjm there aren't much. I might not have found all of them but did tag the once I could locate. :checkbox-yes:

xjm	:cricket:
hestenet (he/him)	I get a feeling that I'm about to send a calendar invite to myself to have a glass of wine and update this later.
xjm	:wine_glass:

heddn	Can @xjm clarify this?
xjm	Referring to https://drupal.slack.com/archives/C7QJNEY3E/p1588776051042900?thread_ts=...
heddn	got it
heddn	automatic updates has a cron job. using ultimate cron, you can set it to run at specific times of the day.
heddn	nothing more, nothing less
xjm	Ahh. Which people who use their own cron config can do anyway if they know what they're doing and their host lets them.
xjm	I guess this is more "phase 3+" territory for when we try to add contrib updates -- that's when batching would become important
heddn	@xjm ultimate_cron.module is a contrib module that lets you pick which hook_crons to run and on what schedule. every 1 minute, only between 2-3 am, etc. I was trying to toot Drupal's horn when I mentioned we can schedule the running of automatic updates (edited)
xjm	I wonder if we want to consider that as a core cron configuration feature
heddn	It would be a nice one. I think it would solve many concerns a single 3 or 6 hour cron run for "everything". It would be a nice, easy feature to add to core.
heddn	https://www.drupal.org/project/drupal/issues/154043 => tagged

heddn	so I can tentatively switch to anything else
xjm	Not "Lucas should implement this"
xjm	Maybe another new thread might be what the next top priority should be? We could even do a poll to get opinions from the list of next things.
heddn	I think dev-master or git clones, or anything but packaged downloads can be avoided at the moment to use CSIG. I'm OK with that.
drumm	Signing for the actual packaged downloads will be straightforward to do, as it will be our 3rd use of signing. But I’d like to understand the practical implications of TUF for what APIs & services Drupal.org provides before jumping into another one.
dts	I need, like, a week hacking with 1-2 other folks to make TUF happen. I'm open to blocking off that week almost at any time.
xjm	The only downside for TUF is that it's reimplementing something we already solved. That said, if it gets us wider support outside Drupal, that's pretty gravy. And folks from the CMS security group seemed interested.
drumm	I think I may have hard at some point that TUF might not affect the existing .csig format. If that’s true, I don’t have any blockers. My very loose understanding is that a lot of TUF is more metadata on how things are validated, not the specific implementation.
dts	It would affect the format, but it's possible to make the transition smooth by relying on the same root trust.
heddn	@dts @drumm @hestenet (he/him) if we wanted to coordinate w/ Peta would could probably get a week virtual sprint on the calendar some time.
hestenet (he/him)	Right now is not the most awesome time for finding a week - we're trying to dig out of being behind.
hestenet (he/him)	But sometime mid-June or later might be possible.
hestenet (he/him)	I like the idea though
(anonymous)	Comment Redacted
heddn	@hestenet (he/him) June is fine by me.
hestenet (he/him)	Currently thinking about week of June 22nd.
heddn	I don't have anything booked that far out. And I'm not traveling much these days :wink:
dts	The Composer folks are amenable to signing support, especially based on the proposals I shared with them around TUF.
heddn	@dts the PoC in https://github.com/heddn/php-signify-composer-integration shows you can already do it NOW, using any hashing/signing system you wish.
dts	BTW, I have much more availability now. I was leading data pipeline work for CovidActNow.org, but I handed off my responsibilities this week.
dts	Oh, nice.
heddn	but only for zip files. the other part if git repo clones.
dts	If it's cloning from a git repo, then it isn't going to go through packaging, anyway.
heddn	and for those... you can be sure no MiM, but not that some malicious person rewrote git history and the git repo is hacked. For that you'd need signed commit messages (I think)
heddn	This is less clear to me and feels like more of an edge case. :shrug:
dts	I would ignore the git clone case for Composer for auto-updates.
heddn	I was going to tacitly ignore it for now as not being low hanging fruit :slightly_smiling_face:
dts	In fact, I'd consider whether we should prohibit releasing a module that depends on a git clone as dependency.
xjm	That's an interesting idea
heddn	Also of note, we also got PHP Signify itself to have automated tests and prove they work on php 5.3-7.4: https://github.com/drupal/php-signify/actions
xjm	Although, we should be signing our commits already and certain core committers have meant to get that sorted for several years cough
dts	Commit signing is generally done on the machine creating the commit, though. I don't believe we can sign commits for users on the server side (without having their keys, of course).
heddn	having signed core commits would be a good first step. along with docs explaining the process so high profile projects (pathauto, token, etc) could start using it too (edited)
dts	If we did sign commits for users, we'd probably need to rebuild the commit, too. It would break fetch/pull
xjm	Right, this would be more a policy/initiative for maintainers
xjm	@dts Wait, why?
xjm	I don't mean signing past commits, only adopting it in the future
dts	Doesn't the hash of the commit include any signing attestations? Or is that entirely auxiliary to the commit itself?
heddn	So, to answer the question earlier about require composer v2, yes. I think it will be very important to only support v2.
xjm	I guess we would say "Composer 2 or tarballs"
xjm	Which should hopefully hit a majority of sites until we stop supporting tarballs.
xjm	By which point Composer 2 will be in wider use :fingerscrossed:
dts	So, I'm still referring to a potential issue for future commits. I believe it requires the GPG signing to happen on the committer's machine.
xjm	Googles "How does git commit signing work"
dts	From Stack Overflow:The GPG signature data then take place in calculating the SHA-1 checksum for the commit to become the commit's hash ID.So, yes, the commit must be signed at commit time if it's going to be signed at all.
dts	https://stackoverflow.com/a/42446250
drumm	And we need to either expose GitLab’s page to put in your PGP info as a maintainer, or sync it over from Drupal.org.
dts	Conversely, if the commit already has a hash assigned, it's not possible to add a signature to it without altering the hash (and breaking the already distributed DAG).
xjm	Right, but that's OK I think. I don't think there's any reason we would need to sign any commits before auto-updates is a core feature, if (as David points out) we need to support it at all)
xjm	I guess the contrib module is the other point of vulnerability
dts	I've just been staying away from any designs that require developers to do key management or signing
drumm	Yeah, commit signing is nice to have, but not really related to automatic updates.
xjm	Circling back, how many sites actually run from git clones these days, I wonder?
dts	Commit signing might make some sense for automatic updates if the update relied on pulling git data.
(anonymous)	Comment Redacted
heddn	@xjm any project that uses a dev-master branch uses a git clone
heddn	w/ composer
dts	I'm also trying to avoid any designs that require a git client on the web servers.
dts	(Even if there is a git client, version differences can be a minefield when the possible versions are old enough.)
heddn	so, please set priorities. Right now I'm focused on the in place updates via composer v2. There's also porting over readiness checks to core. We could also do some work on an A/B FE controller.

xjm	So: Lucas suggests it should be
xjm	For my part, I'm thinking about what this would mean in terms of when we can land support in core. We've put work into making sure D9 is forward-compatible to Composer 2, but won't be able to require it probably until D10.
heddn	Yes, I don't see how we can do it securely unless we have it built-in. I also feel that composer v2 could easily start building on our shoulders if we build a good enough hashing system.
xjm	Since D9 core and d.o still support tarballs, saying you need Composer 2 or a tarball install seems safe and should cover both forward-compatible sites and long-tail BC hopefully
greg.1.anderson	n.b. Drupal 8.8.x does not support Composer 2. So, if you require Composer 2, then you must get up to 8.9.x / 9.0.x before using auto-updates, if you’ve already switched to Composer.
greg.1.anderson	(Or we backport Composer 2 support to 8.8.x)

heddn	@xjm we also have a plugable system for handling DB updates.
heddn	@dww @xjm another thing to consider, it would be awfully nice if we could collect metadata on an update if it has any db updates included in it. Right now, we don't have that info until we download, install and check if there are any. Which is a little late. Do we have an open issue for this?
xjm	Not to my knowledge -- great idea
xjm	It's pseudo-discoverable with update hook numbering, but not totally because update hooks get committed to the forward branches as well, so an update might be new in both 9.1.8 and 9.2.1
xjm	Also the numbering is just convention and not an API
xjm	Post-update hooks have a registry of versions though
xjm	Which is at least half of update hooks these days
xjm	(They do only as of 8.8.3 or so because we realized we didn't know which we could kill for the D9 upgrade path cleanup)
heddn	That doesn't cover contrib either
heddn	and they are the ones most likely to nuke a site accidentally
dww	I thought the post update version registry was only for last removed updates?
xjm	Oh. You're right.
xjm	We could implement it the same way
xjm	Except argh it'd be a data structure change
xjm	Well, definitely worth a core issue in any case
xjm	@heddn I think it's more "are there updates the site has not run" than the updates in the release though
heddn	#3134570: Provide metadata to update.xml that a release has a db update
xjm	Will comment on issue
xjm	Pinned the thread to the channel so we can gather data post-meeting as well
xjm	Also lol @ our distributed opinions

Automatic Updates Initiative meeting on May 7th

1️⃣ What topics would folks like to chat about? Comment and we'll make threads for each.

2️⃣ xjm shared an update from the Wordpress community about their autoupdate efforts - what can we learn from them? https://make.wordpress.org/core/2020/05/02/merge-announcement-plugins-th... (edited)

3️⃣ Core issue organization and setting an issue tag for these issues - the tag should be "Automatic Updates Intiative" -- see thread below for issue search link (edited)

4️⃣ The meta issue badly needs an update - anyone want to volunteer? https://www.drupal.org/node/2940731 (edited)

5️⃣ Are cron improvements mentioned by @heddn in scope for part of this initiative? (Firstly - perhaps summarize what we're talking about here. :slightly_smiling_face: )

Yes sorry, was just trying to understand the problem :slightly_smiling_face:

7️⃣ Should Composer 2 be required for Automatic Updates in Core? Some considerations already being discussed in the 6️⃣ thread above.

We can at least file the issue for the readiness checks even if we don't prioritize it. Architecturally in D8 it will probably want to be plugins. No idea how it's implemented in contrib for D8 atm. (edited)

Comments