This meeting:
➤ Is for core developers, initiative contributors, the Drupal Association and anyone interested in the initiative.
➤ Usually happens every other Tuesday at 1700 UTC.
➤ Is done over chat.
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to
➤ *Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Transcript

0️⃣ Who is here today? Comment in the thread below to introduce yourself and tell us why you are joining us.

effulgentsia :wave:. Alex from Acquia. I'm only here synchronously for the next 15 minutes or so, but will catch up on discussions that happen after that tomorrow some time.
hestenet (he/him) Tim from the DA :wave::skin-tone-3:
tedbow Ted from Acquia :wave:
dts Had a conflict. Joining now to read backscroll. :wave:
TravisCarden :wave: Travis Carden from Acquia. Here to report and field questions on Composer Stager as necessary.
longwave :wave: Dave, core release manager

1️⃣ Do you have any topics to propose for the meeting today? Feel free to propose them in this thread, and then I will give them their own unique threads for discussion. Conversation moving slow? Go ahead and open your own thread in the next numeric order.

xjm Eventually, we will also need a committer review. Coincidentally, xjm may be available for contracting.

2️⃣ Security auditing for PHP-TUF and Rugged is contracted, but the proper start date on the audit work is now mid-OctI will continue pushing to accelerate.This does not block alpha, but may affect beta/experimental deadlines.

3️⃣ Security auditing for AutoUpdates Drupal codeI am still having trouble finding an individual or vendor with availability. Have one lead going, but still waiting to hear.

4️⃣ Deploying Pre-Prod RuggedGoal was to try to do that this week - what does pre-prod rugged mean:It means production like cluster running rugged, step up from our staging setup. It does not mean: fully integrated into packaging pipeline/ready for production use. The AWS outages and some other unplanned work may delay, we will see.

effulgentsia @tedbow, @phenaproxima, @drumm: I'm curious what pre-prod means for the AU module. That module is serving two purposes:A small number of sites are actually using it (not sure if for real or only for testing).It (or a core MR derived from it) needs to start going through the core review process.For #2, we should enable TUF, and I think pre-prod is fine for that. But then how would that affect #1?
drumm I’d expect 2-3 weeks for it to be usable by clients
drumm Op top of discovering unknowns as we get everything set up, backfilling signing will take some number of days
drumm *on
tedbow Right now we only have pre-releases of the 3.x branch so we could change it there.  We can also test this out only on MR so we could at least start testing when ready for clients.

5️⃣ Nils from Packagist/Composer will be at DrupalCon! Coordinating with him to attend our session, and likely present one of his own since Ted's cancellation frees space for another session. (edited) 

xjm Wait I thought we were taking Ted's slot
hestenet (he/him) Sorry - I mean 'the canceled space in the schedule for a session' not the specific timeslot.
xjm Ah!
hestenet (he/him) This is practically a whos-on-first comedy routine :lolsob:

6️⃣ Contrib module: @phenaproxima is working on Rely on TUF-protected resources to determine which updates are availableBasically since Update XML will not be TUF target we cannot rely on this to determine which updates are availableComposer Metadata is TUF protected so we should be able rely on that.Will require Drupal core security releases to be reported back from composer auditFor Project Browser to use this eventually that would need to be all Drupal project security releases (edited) 

drumm I think #3301876: Implement “list security advisories” Packagist/Composer API should be doable this week
tedbow @drumm great!
drumm https://drupal.slack.com/archives/C51GNJG91/p1695325118918529
tedbow @drumm nice!!!! This is for drupal/core and all contrib projects?
drumm Yes
tedbow Great!I think this will be all we need the for the  autoUpdates MVP because it will only support core and we can probably figure out core supported status.Eventually we need to figure out how we can figure out supported status for contrib without update XML
drumm https://github.com/composer/composer/issues/8272 doesn’t look hopeful
tedbow Ok thanks for the link
drumm I suppose we open an issue for packages.drupal.org to either push for that; or conjure some metadata to make unsupported versions look like security issues; or throw it in extra metadata & have the client extensions understand that

7️⃣ We, of course, will also need framework manager, committer review, etc! And xjm may be able to do with funding. (edited) 

xjm We'll also want to secure some time from a framework manager, probably one who has not already been involved in the initiative. larowlan or alexpott.
xjm I think it's important to start planning for this now because I realized none of my clients want to sponsor a 3-month review process that yields one credit.
xjm Past initiatives have gotten blocked for months because committers were not available, and the scope of AU's codebase from TUF through the UI is huge
hestenet (he/him) We have been thinking about setting up one-off bounties for extra hard issues
hestenet (he/him) We have a way to do that.
xjm Like a mini D8 Accelerate -- great idea
effulgentsia Obviously, a huge +1 from me to get a bounty or contract from the DA to @xjm (and also to a framework manager who's not me if one is available) 🙂
hestenet (he/him) Extra credit bounty is actually pretty easy. $$ is always the tough one, but we're at least looking at it
xjm So like, certain issues would secretly get a marketplace ranking bump worth multiple normal issues? Or special treatment on the org page? That kind of thing?
hestenet (he/him) Basically we have a 'special projects' field on org profiles where we can add arbitrary credit. We have internal guideilnes about how to use ofc - but that's the underlying technical capability. And we can annotate it.
hestenet (he/him) So we could say:+50 credits for issue #1235
hestenet (he/him) (This is a relatively new capability - we haven't really used it yet)
xjm That sounds very useful
hestenet (he/him) I'd be very glad to use it here.
Warped Credit in review/audit for each issue discovered would give incentive to sponsoring initial review/audit, since that would likely create the most issues, and is very important.

:8ball: I am working on Fix module so conversion to 11.x core merge request works. We temporarily had a Drush dependencies in the 3.x version of the contrib module we weren’t keeping the core MR up-to-date but now that Add Symfony Console command to allow running cron updates via console and by a separate user, for defense-in-depth is we removed the drush dependencies(was only ever in pre-releases)

xjm @catch @longwave More 11.x aliasing issues :disappointed:
tedbow @xjm not sure what you referring to
xjm @tedbow It's not aliasing issues?
xjm reads issue again
longwave we removed all the alias things so there should be no issues
xjm @tedbow The IS is not helping me here :joy:
longwave there are still issues testing contrib against 11.x when modules (correctly) do not yet declare compatibility with 11.x
tedbow no we just a have conversion script that turns the contrib module into the core version. whenever we don’t run in a  while we have to fix some thing
xjm Ohhh -- can I retitle the issue to "Update merge request process for current HEAD" or something?
tedbow sure
longwave what's the issue with readonly properties? they are readonly but phpstan thinks they aren't?
tedbow FWIW I have it down to 1 fail https://www.drupal.org/pift-ci-job/2766866 (running in testing issue for now)Package Manager will use systems rsync though so we have to install for Drupalci to workWe have a fallback to rsync but currently there are bugs in it.

9️⃣ Related to ^, @xjm, @catch, @longwave: any thoughts on #3385644: [policy, no patch] Consider whether to keep Package Manager and Automatic Updates in a separate repo/package than core in order to facilitate releasing updates to the updater? No need to answer now, just making you aware of the issue.

xjm @quietone also
xjm My kneejerk is "yech
xjm But I will read the issue
xjm Essentially, the problem boils down to that with respect to minor update functionality, any future improvements we make to the updater would take one additional full minor cycle until it's usable, if the updater itself is part of the drupal/core repo.Works as designed IMO.
xjm We would not do minor updates of an external dependency in patch releases either
effulgentsia We wouldn't, but the site could.
longwave what if it was required for a security release?
xjm We would bend over backwards to make it BC as always
xjm As we've done many a time for dependencies we don't control 🙂
xjm To me this just sounds like AU wants to entirely burn the experimental module process to the ground because they're better than other initiatives or something; I have concerns with that aspect too
xjm I guess maybe you're trying to blend the CKE5/JSON:API approach with having something surfaced by core?
xjm I will think about it but initially I have concerns on many sides
longwave we can make exceptions to the backport policy, at least to the currently-supported minor, if needed
longwave If we commit this improvement to 11.2 only (not backport to 11.1 or 11.0)ie. here we could backport the improvement to 11.1 as well if it was considered important enough
xjm Yes, we can backport safe things to experimental betas, and do, when the value outweighs the disruption
longwave 11.0 would be security only by that time and so usually would be left alone
xjm We don't need to add the overhead of an external repo to achieve that discretion
effulgentsia No, that's not the motivation. The updater is unique in that (if part of drupal/core itself) you can't use its functionality until there's a released version that's one higher. For everything else, people can even test a dev branch.
xjm We also consistently backport upgrade path fixes to the security-only branch, so something that's a bug in AU would often qualify
xjm @effulgentsia I don't understand that statement and it does not seem to be covered in the IS -- why?
longwave is it that the updater has to be forward-compatible with whatever version you are trying to update to?
longwave and we obviously can't guarantee that until the future version comes out
tedbow if the fix to the update is only in the latest manner. one of the funcitonalities, updating to the next minor, can never actually be tested
tedbow by anyone manually
catch "If we commit this improvement to 11.2 only (not backport to 11.1 or 11.0), then no one can test it (without applying some version remapping shenanigans) until after 11.3 is released, since you'd first need to update to 11.2 using 11.1's updater and then updating to 11.3 would be the first core minor update that you can perform with 11.2's updater." from the IS
effulgentsia Sorry, need to jump on a work meeting. I'll read up and respond to this thread tomorrow.
xjm I don't understand how that means what Alex said
xjm Or how that's different from other modules
xjm You can always manually update
xjm You don't need autoupdates to update itself when it is experimental
xjm Maybe I should try to understand this when I have more time
effulgentsia Yeah, there's no rush with this
longwave are you just saying that the first version will not update core itself, but future versions might be able to update core, and that would require manual upgrades of core to use that functionality? if so i think that is fine for a minor feature release; we don't expect other features to be backported so why is this one special
longwave it's actually a good thing to announce in a blog post/release note/etc that Drupal 11.2 will be able to self-update to future minors, instead of just saying "this is now available if you update drupal/somepackage"
xjm The thing is they can update core; it's just a config setting
xjm It's just off by default in the MVP
xjm Sorry other way around
xjm The functionality is there for any kind of update you would do; we just configure it to the safest subset to start
xjm Unless the design has changed since I worked on it
tedbow for almost everything else you as soon as it is committed to dev you can check it out and test it manually .that is not true for auto-updates for 2 reasons.if you check out dev we don’t support any updates from a dev versionEven without 1) when the fix to the updater ships in 11.2.1. Nobody will actually know if the complete functionality of the module works until 11.3.0. Because you can’t do a minor update until then. So basically from 11.2.1 until 11.3.0 comes out we may have broken minor updates. We won’t know for sureI am not saying that is good enough reason to put it in its own repo but it does make it different then any other module
xjm It's contrib updates that are dangerous, or minor updates, but a site should still be able to turn them on with code if they are bleeding-edge. But you can always still manually update.
xjm Is there an issue for the dev version thing? It seems like people would need to update from dev versions all the time
tedbow really? I think at that point you could just use composer manually. I don’t think we can safely know your situation at that point
xjm @tedbow I think if we broke updates we would backport fixes; that's like ultra-critical. It's upgrade path.
longwave i don't get how 2. is different whether the updater is in the same repo or not
tedbow it depends which version of core we make the separate repo compatible with. We could keep compatible with the last 2 versions so right away it would possible to use it for a minor update
xjm @tedbow It just sounds like a policy decision that might have been discussed somewhere. I think I would be surprised if I wanted to update to stable but the normal workflow was broken, if that's a production design to not update from dev. But I think it should be discussed in an issue so as not to sidetrack this.
longwave you also can't force a user to update drupal/automatic_updates to get a compatibility fix with 11.3.0 that wasn't anticipated when they installed 11.2.1 and the related version of AU, so you're in the same situation
tedbow @longwave I am not sure if are talking about the same thing.if the updater is part of core, say11.2.3 - is the last drupal version. 11.1 is still getting security releases11.2.3 - adds a change to the UpdaterThe Minor updates functionality cannot be tested in 11.2.3  because there is no 11.3.0 to update toIf the updater is NOT part of core1.0.1 of the Updater is released.1.0.x of the update is compatible with 11.0.x (as far back as far possible in patch verisons) and 11.1.x1.0.1 could be tested to update core from 10.1.LATEST to 11.2.3In gitlab testing we could set up a matrix of testing to updating from all compatible 10.0.x versions to 11.2.3. This would ensure know that core versions would be compatible with the updates not just generic build testing
longwave you could still have that matrix by patching 11.1 and ensuring it can update to 11.2 even if we never released 11.1 with that committed
longwave you still can't guarantee an update to 11.3 will work until 11.3 is released either way?
longwave in fact I think there would be some benefit in allowing a test to update a minor to -dev HEAD to ensure that we haven't broken anything in HEAD
tedbow no you can’t be user.but you could make a test with composer aliases to make sure that 11.3-dev works as 11.3.0
tedbow you could do that even it is part of core
longwave indeed
tedbow so yeah we should explore what is possible. We will probably need new kinds of the test than what we have now
tedbow allowed by gitlab but also because we have new need with an updater
longwave a build test might be able to do it if it has a full checkout of the repo, ie. both the from and to branches available
tedbow sure we could probably do anything.perhaps as pre-steps in gitlab we could create composer projects for every version core we want to update from
longwave or just allow the code to update to dev releases but only allow it to be used in tests? unless there is some other limitation on that
xjm Is the dev release thing one of the readiness checks?
tedbow yes
xjm OK so a decision we made several years back and we should dig up and review why 🙂
tedbow k, will look into it.

🔟 Composer Stager is now ready for RC pending code review by Core maintainers.

hestenet (he/him) Excellent!
TravisCarden https://github.com/php-tuf/composer-stager/issues/281

1️⃣1️⃣ Should automatic updates support update from dev versions of core

tedbow @xjm I think part of the problem is that if you are running dev that you could be running any commit on that branch or could have run any commitThis includes commits that were reverted. So what if one of those include a post_update hook that we reverted but you already ran it (edited)
tedbow but that post_update was never part of a release
tedbow what if you just happen to run a commit that had bug that corrupted you config or content.Of course that is possible with a release too. but it opens a lot more points concern if we don’t just have to think about releases but every commit
xjm Ahh, because we don't provide BC until the next tag
longwave I think no, except in tests where we should be able to try upgrading one branch to the next to prove the next release will work
xjm I mean dev is unsupported, 100% unsupported, but loads of people use it anyway
longwave And if you only allow upgrading to a dev release (but not from), then you have a one way door
longwave Once you have upgraded you can't do it again
tedbow to many updating to or from dev just is opening a to many unknowns. When updates go wrong people will blame the updater not the fact that they were updating from dev version.I think that fact that we are building UI updater is signally to user we have some confidence that there updates will go wellif you are coming from dev version I would have near 0 confidence (edited)
tedbow we are going have enough problems with edge cases with people updating from 1 actual release to another.if get a track record of those going well then I think could consider expanding what we support.some one can still make auto_updates_bad_judgement  and remove our rules  and support to and from dev releases.
hestenet (he/him) Ha

Participants:

xjm, effulgentsia, drumm, tedbow, hestenet, Warped, longwave, catch, TravisCarden

Comments

hestenet created an issue. See original summary.

hestenet credited catch.

hestenet credited drumm.

hestenet credited effulgentsia.

hestenet credited longwave.

hestenet credited tedbow.

hestenet credited TravisCarden.

hestenet credited Warped.

hestenet credited xjm.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Issue summary: View changes
xjm’s picture

xjm
she/her
Primary language English
commented

Amending attribution.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.