Automatic Updates Initiative meeting on July 27th, 2021 [#3224708]

This meeting:➤ Is for core developers, initiative contributors, the Drupal Association and anyone interested in the initiative.➤ Usually happens every other Tuesday at 1700 UTC.➤ Is done over chat.➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!➤ Has a public agenda anyone can add to: #3224708➤ Transcript will be exported and posted to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.0️⃣ Who is here today? Comment in the thread below to introduce yourself and tell us why you are joining us.

dts	waves
dts	Hi all! David here from Pantheon/Sec Team.
TravisCarden	:wave: Travis from Acquia
tedbow	:ocean: Ted from Acquia, Ithaca, NY, USA
hestenet (he/him)	Tim from the DA in Portland, OR :wave::skin-tone-3: - Thanks for running the agenda today @dstol :blue_heart:
effulgentsia	:wave: Alex from Acquia
xjm	:wave: half here
JayKandari	:wave: Jaideep from QED42. IST timezone.

1️⃣ Do you have any topics to propose for the meeting today? Feel free to propose them in this thread, and then I will give them their own unique threads for discussion.

tedbow	couple items in sandbox project for composer updates[#3225508]
dstol	@tedbow want me to break one and two into 2 and 3 in the main thread?
hestenet (he/him)	Topic: The DA has received at least one bid to help us with the TUF server side implementation - and hopefully can sign an agreement within 2 weeks to get that on track again.
tedbow	@dstol yes pleaseanother one how to move work from the sandbox to contrib to core
dstol	:thumbsup: coming right up

2️⃣ changing readiness checker to event instead of tagged services #3225099: Change Readiness checker services to event subscribers to allow checking at different stages (edited)

tedbow	This was mainly done because we want to checks at least 3 points in the update processcron regular readiness checks. an update is not happening but should inform the admin if there will likely be problem updating(example file permissions)an update is actually startingan update has been staged but not committedonly 1 needs the logic in our ReadinessCheckerManager which handles not running the checks too often and other cases2 and 3 can be much simpler events but it would great if modules can 1 way to respond to all 3.
dts	Are we planning to stage updates for site admins to apply -- even ones that we won't automatically apply?
tedbow	this would be for attended updates.but also for automatic ones we may want to reject certain updates based on things we can only tell once it is staged see #3179266: Automatic Updates that require updates to contrib and custom extensions should be rejected
tedbow	@dts but you mean if we have one that doesn’t pass the apply automatically criteria ?
dts	If it has a schema change, for example.
dts	Or, would we only stage it after some initial admin interaction? Or is this all out of scope right now?
tedbow	I personally think if has a site has auto updates enabled and we reject an update that is staged we should just clean() the update and notify the admin with a link to page to do it manually.mostly because we don’t know when admin would actually get to it so the update might be stale at that point
dts	I wasn't sure if we'd continue gardening the staged data to be fresh.
tedbow	I guess I don’t see the benefit of the continue gardening logic vs just telling the admin to apply it from scratch. the 2nd way seems simpler and less error prone
dts	It's sometimes nice to see that an update is downloaded and ready-to-install. I know my desktop machine does that, and Windows does, too. I do not feel strongly about this. I was just curious about the plan.
tedbow	we haven’t talked about it. we have to attended update anyways because some admins won’t want anything to happened unattended. so it seems as mvp it unattended fails we could just have an error and link to attended form

3️⃣ end-to-end functional test of attended updates #3225508: Create an end-to-end test of updating core which can be run offline

tedbow	@phenaproxima (he/him) is working on this but we need to debug because there is currently a problem with the test when your computer is completely offline related to update xml
phenaproxima (he/him)	Yeah, working on debugging this, but it could take a while.

4️⃣ how to move work from the sandbox to contrib to core

tedbow	related to this issue [#3220205]not sure there is much to report here(I have to read the most recent comments)but there was an idea of committing it to core as alpha experimental and then having a script push this to the contrib module.this might actually be more complicated then we first thought because we had planned that the core and contrib version would have different machine namesautomatic_updates vs auto_updates . There are problems I think with moving a module from contrib with exact same machine name
dts	I'd consider naming some of this work something closer to package_manager than just something related to auto-updates. We may be delivering auto-updates as our main goal, but most of our work and functionality here is in GUI package management.
dts	My understanding is that we intend to leverage this work to support things like module installation.
tedbow	yeah we could change the name. but it still holds it likely won’t be the same as contrib so a script to take core version and push back to contrib is not super simple
effulgentsia	If we change the name, do we need a contrib module at all? Could we just then have the sandbox and core, with the same name in each?
tedbow	we could be that would mean also couldn’t get it in the hands of users sooner. and if it doesn’t get into 9.x at all then only 10.x users would have it
effulgentsia	Yeah, that's a good point.
effulgentsia	Does anyone remember what the specific problems were with having a core module with the same name as a contrib one? Especially, for example, if we make the contrib one for D9 only and the core one in D10 only, would that alleviate whatever those problems were?
tedbow	I think @xjm has mentioned problems related to the json_api experience (edited)
xjm	They are related drupal.org not being able to resolve which drupal/packagename package is being included in composer
effulgentsia	Ah, right!
xjm	There should be a whole issue about it in the queue somewhere and with a note to write handbook documentation about it
xjm	I can probably dig the link out in a bit
xjm	I think we still need buy-in from the security team members who have pushed back on having a contract alpha without signing and see if they agree with it based on the updates on the issue sense
effulgentsia	If we want to name the core one something like package_manager, would it make sense to name a new contrib module something like package_manager_contrib?
xjm	It should probably also have committer roll sign offs
xjm	@effulgentsia I think there is something about prefixing instead, will check Also /cc @mixologic Knows the most about this problem space
dts	I think we still need buy-in from the security team members who have pushed back on having a contract alpha without signing and see if they agree with it based on the updates on the issue senseI think relying on HTTPS is extremely non-risky if we include some client-side checks about capability. The issue is that these checks will disqualify a good number of sites from using these features until there's the signed data alternative that will work universally.
xjm	Let’s update the issue summary with our specific plan in that regard, and then ask those team members for a re-review?
dts	Yes.
dts	Is it the issue linked in this thread?
tedbow	yep top 1 I linked
effulgentsia	The issue is that these checks will disqualify a good number of sites from using these features until...I this that's ok, maybe even desirable. The purpose of the alpha is to get some early testing, but not necessarily from people with hosting that lacks https, etc.
effulgentsia	Obviously we'll want testing from those people too, but that's what a beta can provide.
dts	A lot of hosting will have HTTPS but outdated. Like, they'll have PHP linked to a version of OpenSSL that can't do TLS 1.2, or they'll have an outdated CA chain. The main risk of just trying to use the HTTPS on a box is that, if the box has outdated OpenSSL, it may be possible to exploit it even if our side is modern. So, I'd propose doing some client capability checks before being willing to leverage local HTTPS client config.
effulgentsia	Sure. That's the "etc." part of my comment :slightly_smiling_face:
dts	Ah.
dts	What I'm proposing is stronger than what Composer + Packagist does today.
tedbow	“etc” covers anything you might mention in the future :joy:
dts	I'll comment on the issue.
tedbow	@dts thanks
effulgentsia	So for @xjm's other point about process for helping ensure that what's in the contrib module has essentially passed most/all core gates, so that getting it from contrib to core is smooth... Her idea is to have core committers required to sign off on any commit to the contrib project. Another idea that I floated is what if we commit to core first (as alpha, and each patch that keeps improving it but still keeps it at alpha-stability), and then only sync what's in core to contrib (rather than from contrib to core). The purpose of the contrib project then would just be a way for people to get their hands on the alpha-level package_manager code, even if they're on a tagged release of core.
effulgentsia	Thoughts?
dts	I'm a fan of going to core quickly as long as (1) we're not worried about it slowing development and (2) we can accept a milestone for core inclusion that's reachable quite soon (rather than have code in core that's not even an alpha).
effulgentsia	As a reminder, alpha in core means it's in the core dev branch, but never in a tagged release. E.g., once 9.4.x is opened (which is before 9.3.0-alpha1 is released), it gets removed from 9.3.x.
effulgentsia	However, there are still quality/review gates that happen for the initial commit to core, and each subsequent patch.
effulgentsia	So what I'm proposing is... 1) iterate in a sandbox. 2) as soon as what's in the sandbox is core-alpha-worthy, post that as an issue/MR to core. 3) once it lands in core, sync it to a contrib project.And repeat that sequence for each subsequent unit of work.
dts	#3220205: Proposal: As pathway to Drupal core use new 2.x.x branch for Composer Based Updates #comment-14175057
tedbow	to clear the current plan we are working on is working in this sandbox and then once we the MVP moving it over in merge request chunks to somewhere(core or contrib) for core committer reviewso hopefully that will speed up development speed. and make the core reviews not huge.also getting the whole mvp in the sandbox first is already helping shape the architecture rather than just assume what it is going to be based on creating bit by bit in core first (edited)
effulgentsia	I don't know if my proposal is any good or not, so feel free to poke holes in it, but the key feature in it is the separation between sandbox and contrib. In other words, sandbox is for developers to work quickly on something before it lands in core. And contrib is for testers to test what's already landed in core, but without waiting for it to make it into a tagged release of core. (edited)
mixologic	Hi, was out yesterday, but is the contrib potentially intended for use by people who have an earlier version of core where it is not supported? The namespaces being different is pretty crucial with the current design of packages.drupal.org, but if it is just for testing, then would it make sense to give it a 'testing' namespace?

5️⃣ The DA has received at least one bid to help us with the TUF server side implementation - and hopefully can sign an agreement within 2 weeks to get that on track again.

dts	Any chance I can help review the bid?
hestenet (he/him)	Yes please.
dts	I'm less concerned with the commercial terms than ensuring there's confidence that the bidding org is likely to succeed given their capabilities and defined project scope.
hestenet (he/him)	Exactly, yeah
hestenet (he/him)	Fwded you the initial email
hestenet (he/him)	@dts Any thoughts on that email?
dts	Oh, of course. They seem very well qualified.
hestenet (he/him)	Great - I'll plan to move forward then.