[policy, no patch] Determine to which module the new security advisory functionality should be added

That's a nice write-up Ted. Thank you.

Regarding:

It is not best practise to have Update module enabled on production. This would mean we either encourage people to do it anyway to get the new security advisory functionality or accept most sites will not get these critical security announcements

IMO the same site admins who disable Update module are, or ought to be, the same admins that track security updates via other channels. All in all I would consider that yet another reason to use the Update module for this effort.

Also, IMO, the Update module is the natural place for these messages. Can we get a hyperlink to evidence about the large portion of the Drupal community who do not consider it a best practice to have Update enabled? Obviously, in-place updates in their original form are from a bygone Internet era, but its reporting feature works just fine.

There's some relevant discussion in #2869592: Disabled update module shouldn't produce a status report warning.

My primary concern with putting this in update.module is that so many sites already have the module disabled, and therefore those sites are unlikely to ever get the PSA warnings. The PSAs are ten times rarer than average security releases and ten times as serious.

Another aspect is that the availability of updates is discoverable by lots of other means (e.g., Packagist/composer), but d.o is the only source for the highly critical PSAs that will end up in the feed.

Re #4 The update module alerting feature has been recommended by security team members at DrupalCon, in contradiction to the mysterious people who consider using it a "bad practice".

I don't think anyone was saying it's a bad practice, only that disabling it in production is considered best practice by many agencies and organizations with any kind of deployment workflow. It's of course best practice to keep it on in prod if you don't have a staging site or deployment workflow. The related issue documents scenarios where having update.module on is not desirable.

Using update module to download modules is not best practice, but just having it enabled for update notifications is fine and should be encouraged. Checking for updates happens on cron so it is not really a performance hit. If you're running a multisite with 2,000 sites and exactly the same modules installed on each, then you would want a centralised update reporting system and to disable update status (rather than checking the same information 2,000 times in order to make a single code update), but the notification we're adding here would also not be any use to you in that situation either - since the person looking at the status report would have no control over the code base to get that update applied.

Really all the reasons given in #2869592: Disabled update module shouldn't produce a status report warning to disable update status are referring to either some kind of 'hosted Drupal' situation, or another situation where you would also not benefit from a notification in the status report.

So I don't see any good arguments to discourage people from having update status installed. If you have an extreme edge case where you don't want it installed, then you've already made yourself responsible for finding out about updates via some other method and the same applies for PSAs.

This doesn't mean that it shouldn't be shown in system module - maybe we want it there as a failsafe just in case, but then I can see 'hosted Drupal' sites wanting to disable it there too for all the same reasons as given in the other issue.

If it does get added to system, there will definitely be something like a config-only system.disable_psa_feed or something for it to be disabled in those scenarios where it's desirable for a HSP or for the privacy-paranoid who don't want their site ever making requests to d.o. It is much lighter weight as it's a single feed rather than one per project.

There's maybe a followup as to whether the module uploader feature should be separate from the update feed information in the first place, but not in scope here.

For me, the cons of having this logic in update.module are:

• The current description that people see on admin/modules for the Update Manager module is: Checks for available updates, and can securely install or update modules and themes via a web interface.. However, according to this issue's summary, the advisories being discussed here include releases that are coming soon (they are not "available updates" yet) and PSAs that need action (also not "available updates"). While we can change the description of update.module if we want to, people have already decided whether or not to enable it based on its current description.
• People might have it disabled if they have privacy reasons for not wanting to report to drupal.org which versions of which modules they are using. Allegedly the data collection is anonymous, but servers can read the requestor's IP address, and some privacy-focused site owners might not trust drupal.org to not retain that information.

Therefore, the pro of putting the fetching of these new advisories into system.module would be:

• People who have disabled update.module for privacy reasons will by default be opted in to these advisories, and the fetching of these advisories does not involve telling drupal.org which modules you're running.

However, the con is that site owners who disabled update.module for privacy reasons, and who not only don't want to inform drupal.org about which modules they're using, but also don't want to inform drupal.org about the fact that their IP address is running Drupal at all, will now have an additional setting that they need to set somewhere in order to opt out of pinging drupal.org for these new advisories.

Although I don't like the idea of making people who have already opted out of pinging drupal.org (by disabling update.module) to have to do so again once they update Drupal core to whichever release includes this new functionality, I can see the argument that it's better to annoy these people than to fail to deliver these highly critical advisories to people who could have benefitted from them.

So on balance, I'm +1 for putting it into system.module, and I hope that if we do so, then our release notes about it are sufficiently visible to the privacy-focused site owners, so that even if they're annoyed about having to change a setting somewhere, they're at least not left feeling like we tried to trick them or that we're trying to violate the privacy preference that they've already made.

Tagging for product manager review, since how we respect or balance people's privacy needs seems to me like the kind of thing that needs product management input.

Also, again, there will be a non-UI config option to disable it and a release note about that option if it lives in system.module.

I basically land exactly where @effulgentsia landed, but also think it's a good idea to at least try to get perspective from actual impacted users on this, so made an attempt at https://twitter.com/webchick/status/1362569962695446530

For example: https://twitter.com/j_brockbank/status/1362577441613570052 raises an interesting point about distributions. It's common for an organization to make a special "flavour" of Drupal for their university/enterprise/etc. and want to centrally manage security updates themselves vs. having every individual department / office / etc. panicking and doing them as one-offs (or overwhelming the IT help desk with "OMG" tickets).

Is a distribution author able to inject a settings.php override to turn off this behaviour? I guess only by hacking core...

Is a distribution author able to inject a settings.php override to turn off this behaviour?

Yes. It’s toggled by a config setting, as per @xjm in #13. A distro could easily turn it off out of the box.

Ok great, that's good to have confirmed!

Second point https://twitter.com/Sam_152/status/1362583789008867328

Is there some smarts built into this notification system so it won't alert you if the highly critical is in, say, the REST module, and you don't have REST module turned on?

(This is not a deal-breaker, I'm just curious.)

Is there some smarts built into this notification system so it won't alert you if the highly critical is in, say, the REST module, and you don't have REST module turned on?

From what I can see (in https://git.drupalcode.org/project/drupal/-/merge_requests/284/diffs#bb5...), it will show the alert if the module is present at all, regardless of whether it's turned on. But if you don't physically have the module in your code base, any alerts for it won't show up.

For whatever my opinion's worth, this makes sense. If I understand correctly, these advisories are only supposed to be for rare, Drupalgeddon-level events that should scare the pants off of everyone. Lesser security issues won't show up in the feed at all. (@xjm or @tedbow could confirm or deny that understanding.) But even then, the alerts can be turned off by a config switch.

it will show the alert if the module is present at all, regardless of whether it's turned on

Correct. Per #4 in "Remaining Questions: Completed" section of the issue summary of #3041885: Display relevant Security Advisories data for Drupal:

1. Certain remote code execution vulnerabilities might not require the module to be installed.
2. A module could be installed at any point. A site admin should probably know that a critical security release is forthcoming in the next week so they can make an informed decision if they want to install a module or maybe wait until the release

Cool, thanks.

It's still early, but indicators in that thread are that people who shut off Update Status do it less because of privacy concerns, but far more often because they want to track updates themselves and apply them on their own schedule (and after ample testing) vs. needlessly freaking their users out and getting superfluous support requests. Which makes sense. It also follows that these folks would prefer keeping all security-related notices together (in Update Status) so they can treat them the same.

Then again, this use case also implies a decent level of sophistication, and one presumes they could easily apply a config override for such extra notification, esp if it's well-documented how to do so in the change record.

And in the case of a site without a dedicated IT team, you actually want to freak the users out, so that they contact their sysadmin/contractor/niece ;) or whoever is doing the updates.

So still leaning towards System module, but it's interesting to hear stories from "in the field" as to what the use cases are and why.

If a site is extremely security conscious and wants to control what it is talking to then it should be using an outbound proxy rather than relying on what modules are installed. It's hard to review all code for possible outbound http requests.

Currently I would put it into update module, but at least #2059375: Update notification messages as an option in settings should be fixed first.
Later I would put it into a separate "lower level" module like the one described in #3199472: Improved data sharing with d.o.

Or keep it in update, if update would have much more improved configuration options, like disable everything except security advisory.
That way it would be best practice again, to have update module enabled on all sites.

Ok, based on the comments at https://twitter.com/webchick/status/1362569962695446530 ... folks have varied reasons for wanting to shut these messages off, but for those paying active attention, we are providing a mechanism for them to shut these messages off, and for those NOT paying attention, system module rightly freaks out people to bug their sysadmin, I think we are good to put this in System module, providing we document well how to work around this behaviour for those managing their own updates.

#2059375: Update notification messages as an option in settings (or something even more robust than that) would also be good to do, since this is a fairly common request.

Based on the above feedback, I think system is the right choice (with the config option to disable it documented in the release notes). I think between myself, @effulgentsia, and @webchick we also have the full trio of committer role signoffs, so marking RTBC.

Given the feedback here and on Twitter, I also think we might want to reconsider what is shipped in update module in general. Security release notifications are relevant to lots of sites, but many sites would not want a field in production that lets module code get downloaded to the codebase. I imagine some site owners might want to keep that out of production entirely (the same way some sites remove Views UI from production deployments' codebase to prevent views from ever being altered in production). Furthermore, the monthly notifications for every bugfix release are excessive or annoying, so some sites might only want to hear about security releases. Finally, it sounds like we need to work on the module UX in general. These things might all affect how we build the Autoupdates UI as well.

All of those things are not in scope here, so tagging "Needs followup" to capture the feedback for later improvements.

Crediting discussion participants.

+1 to putting this in system, FWIW.

Re: #25

Security release notifications are relevant to lots of sites, but many sites would not want a field in production that lets module code get downloaded to the codebase. I imagine some site owners might want to keep that out of production entirely (the same way some sites remove Views UI from production deployments' codebase to prevent views from ever being altered in production).

That was as true when we converted Update status (D6) into Update manager (D7). So there's a poorly named, and even more poorly advertised kill switch for all the "Update manager" parts: allow_authorize_operations. E.g. this section from default.settings.php:

/**
 * Authorized file system operations:
 *
 * The Update Manager module included with Drupal provides a mechanism for
 * site administrators to securely install missing updates for the site
 * directly through the web user interface. On securely-configured servers,
 * the Update manager will require the administrator to provide SSH or FTP
 * credentials before allowing the installation to proceed; this allows the
 * site to update the new files as the user who owns all the Drupal files,
 * instead of as the user the webserver is running as. On servers where the
 * webserver user is itself the owner of the Drupal files, the administrator
 * will not be prompted for SSH or FTP credentials (note that these server
 * setups are common on shared hosting, but are inherently insecure).
 *
 * Some sites might wish to disable the above functionality, and only update
 * the code directly via SSH or FTP themselves. This setting completely
 * disables all functionality related to these authorized file operations.
 *
 * @see https://www.drupal.org/node/244924
 *
 * Remove the leading hash signs to disable.
 */
# $settings['allow_authorize_operations'] = FALSE;

Furthermore, the monthly notifications for every bugfix release are excessive or annoying, so some sites might only want to hear about security releases.

There's a setting for that, too. ;)

Finally, it sounds like we need to work on the module UX in general. These things might all affect how we build the Autoupdates UI as well.

Definitely, but I'm not sure how actionable that is as a follow-up. I'm guessing a "[META] Improve the UX of Update Manager" issue won't get very far on its own.

Maybe "rename the update manager killswitch and make it more obvious how to use it" as a start? Any specific suggestions are most welcome.

Thanks!
-Derek

Thanks @dww.

The screenshotted option for configuring notifications is only about emails. I'm referring to the user interface.

+1 for system module.

This is one of those things that (as the IS notes) is infrequent enough, but when it does happen, is critical enough, that any responsible site owner should know about and should (within reason) have the least amount of opportunities to accidentally turn this off without realising what they are doing.

We've implemented the system.module decision in the main issue, so all that's left here is to file a followup as per #24 and then this can be marked fixed. Thanks everyone!

I've opened the follow-up for update module here, marking this fixed. #3204878: Separate security release notification and code update functionality in update status module

Thanks, @catch. I also opened #3205746: Group the available updates report by status, make each collapsible to address @xjm's point in #27 about the UI being more obvious about security releases vs. regular. I think that's a relatively simple approach with a potentially big UX improvement - reviews / comments welcome.

Other possible issues to use as follow-ups:
#1094018: Improve relevancy of security update notifications within your Drupal site
#2969712: Update module improvements

If anyone else has a specific suggestion, please search / open as needed. Removing the tag.

Thanks!
-Derek