symfony/mime and symfony/var-dumper versions are on 5.1.7 (not an LTS) and therefore have gaps in security coverage relative to Drupal minors

Added @Eric_A's solution from the other issue too.

I confirmed that this issue is still outstanding. We're on 5.0.7 and the latest release is 5.0.8. We went into beta with this so I'm hesitant to downgrade it at this point.

We have an agreement with Symfony wherein we have two Security Team members with access to their private queue, in order to provide security releases that affect Drupal after their non-LTS minors' security support has ended. Originally that plan was not going to kick in until we tried to adopt 6.0, but it looks like we have a scenario for it here. So I guess we need to keep an eye out for security issues affecting those two components.

Meanwhile, we'll want to make sure these get updated to 5.1. It's potentially worth doing during RC -- normally this would be a beta-deadlined change, but the easy security coverage probably outweighs any disruption.

Cleaning up pre-9.0.0 beta target issues.

We should also actually discuss this issue again now. I'll ping the other committers about it.

After #3055193: [Symfony 5] The "Symfony\Component\HttpFoundation\File\MimeType\MimeTypeGuesser" class is deprecated since Symfony 4.3, use "Symfony\Component\Mime\MimeTypes" instead. we explicitly reference symfony/mime in core and that will become a proper hard dependency in Drupal 10. So there is no real issue adding it to composer.json now if we wanted to.

We're at 5.1.7 - atm. I guess that's the other possibility keep these up-to-date with each minor release.

It would be great if that was the only issue, but Symfony 5.1 is only supported until January 2021, whereas 9.1 will be supported until November 2021 - so we have a nine month gap in security coverage (with the fallback that Drupal security team members are now allowed to help backport security fixes to unsupported versions of Symfony components we use).

If we can get onto Symfony 5.2 during November - i.e. during beta or rc, then it's slightly better, but still only gets us security coverage to July 2021 - if we're going to stick to the minor releases we should definitely do that, three months is very different to nine months.

This is going to be a general problem if we're able to jump to Symfony 6 in Drupal 10, so it's a question of whether we want a taste of it now or stick to Symfony 4 until we get there.

It is quite tempting to try to stay on Symfony 5 as a kind of tracking-Symfony-6 taster with these two components to see how it goes. And will give us a smaller jump to Symfony 6 when we get there.

Assuming we get onto Symfony 5.2, we only have to deal with Symfony 5.3 and 5.4 then we're on an LTS again anyway. But of course if we go that direction it could bite us later.

Presumably, if they're being kept up with the Symfony release schedule, the 5.2 versions will be released in November. It'll be after our beta, but around the time of our RC. Would we be comfortable adding a release note to the beta that these might receive a further update prior to RC, and then tagging our RC with the 5* Symfony things bumped to 5.2?

Meant this one.

There are also beta tags that we could add in the alpha for testing, if we decide to go that route: https://github.com/symfony/symfony/releases/tag/v5.2.0-BETA2

The Symfony PHPUnit bridge is also affected. Yes, it's only a dev dependency, but we ship those in dev tarballs so it's something to pay attention to.

Would we be comfortable adding a release note to the beta that these might receive a further update prior to RC, and then tagging our RC with the 5* Symfony things bumped to 5.2?

I would be yeah.

So, Symfony 5.2 is going to make things a bit complicated. symfony/var-dumper and symfony/phpunit-bridge are fine, they just bump straight up. However symfony/mime adds a number of new dependencies, including symfony/serializer:^5.2 - but we currently ship with 4.4.15.

The good news is that the existing MIME tests seem to pass locally, so let's see how this fares on testbot.

The full set of dependency changes are:

$ composer-lock-diff --no-links
+-----------------------------------+---------+--------------+
| Production Changes                | From    | To           |
+-----------------------------------+---------+--------------+
| symfony/mime                      | v5.1.7  | v5.2.0-BETA2 |
| symfony/serializer                | v4.4.15 | v5.2.0-BETA2 |
| symfony/var-dumper                | v5.1.7  | v5.2.0-BETA2 |
| phpdocumentor/reflection-common   | NEW     | 2.2.0        |
| phpdocumentor/reflection-docblock | NEW     | 5.2.2        |
| phpdocumentor/type-resolver       | NEW     | 1.4.0        |
| symfony/deprecation-contracts     | NEW     | v2.2.0       |
| symfony/polyfill-intl-grapheme    | NEW     | v1.20.0      |
| symfony/property-access           | NEW     | v5.1.7       |
| symfony/property-info             | NEW     | v5.1.7       |
| symfony/string                    | NEW     | v5.1.7       |
| webmozart/assert                  | NEW     | 1.9.1        |
+-----------------------------------+---------+--------------+

+-----------------------------------+--------+--------------+
| Dev Changes                       | From   | To           |
+-----------------------------------+--------+--------------+
| phpdocumentor/reflection-common   | 2.2.0  | REMOVED      |
| phpdocumentor/reflection-docblock | 5.2.2  | REMOVED      |
| phpdocumentor/type-resolver       | 1.4.0  | REMOVED      |
| symfony/phpunit-bridge            | v5.1.7 | v5.2.0-BETA2 |
| webmozart/assert                  | 1.9.1  | REMOVED      |
+-----------------------------------+--------+--------------+

Fixed RequestHandlerTest. DecoderInterface::decode() no longer accepts NULL as $format, but for the purposes of the test double this used to be OK. Instead we tell it that we are actually sending JSON and it works again.

I think the ComposerProjectTemplatesTest fails are false positives in some way.

The test error messages are:

Dependency resolution completed in 0.000 seconds
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for drupal/core-recommended ^9 -> satisfiable by drupal/core-recommended[9.1.99].
    - drupal/core-recommended 9.1.99 requires symfony/mime v5.2.0-BETA2 -> no matching package found.
  Problem 2
    - Installation request for drupal/core-dev ^9 -> satisfiable by drupal/core-dev[9.1.99].
    - drupal/core-dev 9.1.99 requires symfony/phpunit-bridge ^5.1.4 -> no matching package found.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
 - It's a private package and you forgot to add a custom repository to find it

Read https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Do we need to add @beta to the end of the beta version requirements to reinforce that we're overriding the minimum stability for these beta packages? Also, should the PHPUnit bridge also be specifying the beta?

Symfony 5.2.0-beta3 is out. Let's try ^5.2@beta as the version constraint and see if that makes the test any happier. Unsure why we aren't getting e.g. symfony/property-access 5.2.0-beta3, maybe prefer-stable is preventing that.

+-----------------------------------+---------+--------------+
| Production Changes                | From    | To           |
+-----------------------------------+---------+--------------+
| egulias/email-validator           | 2.1.22  | 2.1.23       |
| symfony/mime                      | v5.1.8  | v5.2.0-BETA3 |
| symfony/serializer                | v4.4.16 | v5.2.0-BETA3 |
| phpdocumentor/reflection-common   | NEW     | 2.2.0        |
| phpdocumentor/reflection-docblock | NEW     | 5.2.2        |
| phpdocumentor/type-resolver       | NEW     | 1.4.0        |
| symfony/deprecation-contracts     | NEW     | v2.2.0       |
| symfony/polyfill-intl-grapheme    | NEW     | v1.20.0      |
| symfony/property-access           | NEW     | v5.1.8       |
| symfony/property-info             | NEW     | v5.1.8       |
| symfony/string                    | NEW     | v5.1.8       |
| webmozart/assert                  | NEW     | 1.9.1        |
+-----------------------------------+---------+--------------+

+-----------------------------------+--------+--------------+
| Dev Changes                       | From   | To           |
+-----------------------------------+--------+--------------+
| phpdocumentor/reflection-common   | 2.2.0  | REMOVED      |
| phpdocumentor/reflection-docblock | 5.2.2  | REMOVED      |
| phpdocumentor/type-resolver       | 1.4.0  | REMOVED      |
| symfony/phpunit-bridge            | v5.1.8 | v5.2.0-BETA3 |
| webmozart/assert                  | 1.9.1  | REMOVED      |
+-----------------------------------+--------+--------------+

Forgot symfony/var-dumper.

+-----------------------------------+---------+--------------+
| Production Changes                | From    | To           |
+-----------------------------------+---------+--------------+
| egulias/email-validator           | 2.1.22  | 2.1.23       |
| symfony/mime                      | v5.1.8  | v5.2.0-BETA3 |
| symfony/serializer                | v4.4.16 | v5.2.0-BETA3 |
| symfony/var-dumper                | v5.1.8  | v5.2.0-BETA3 |
| phpdocumentor/reflection-common   | NEW     | 2.2.0        |
| phpdocumentor/reflection-docblock | NEW     | 5.2.2        |
| phpdocumentor/type-resolver       | NEW     | 1.4.0        |
| symfony/deprecation-contracts     | NEW     | v2.2.0       |
| symfony/polyfill-intl-grapheme    | NEW     | v1.20.0      |
| symfony/property-access           | NEW     | v5.1.8       |
| symfony/property-info             | NEW     | v5.1.8       |
| symfony/string                    | NEW     | v5.1.8       |
| webmozart/assert                  | NEW     | 1.9.1        |
+-----------------------------------+---------+--------------+

+-----------------------------------+--------+--------------+
| Dev Changes                       | From   | To           |
+-----------------------------------+--------+--------------+
| phpdocumentor/reflection-common   | 2.2.0  | REMOVED      |
| phpdocumentor/reflection-docblock | 5.2.2  | REMOVED      |
| phpdocumentor/type-resolver       | 1.4.0  | REMOVED      |
| symfony/phpunit-bridge            | v5.1.8 | v5.2.0-BETA3 |
| webmozart/assert                  | 1.9.1  | REMOVED      |
+-----------------------------------+--------+--------------+

Curiously requested a retest because the first 9.1.0 beta is available now. But the patch doesn't apply anymore.

Symfony has removed the new dependencies from symfony/mime, which really helps us out here.

https://github.com/symfony/symfony/pull/38888

The new lock diff is

+-------------------------------+--------+------------+
| Production Changes            | From   | To         |
+-------------------------------+--------+------------+
| symfony/mime                  | v5.1.8 | v5.2.0-RC1 |
| symfony/var-dumper            | v5.1.8 | v5.2.0-RC1 |
| symfony/deprecation-contracts | NEW    | v2.2.0     |
+-------------------------------+--------+------------+

+------------------------+--------+------------+
| Dev Changes            | From   | To         |
+------------------------+--------+------------+
| symfony/phpunit-bridge | v5.1.8 | v5.2.0-RC1 |
+------------------------+--------+------------+

RC2 is now available; unfortunately, it looks like they might not release 5.2.0 until right before 9.1.0 ships.

The test fails are still:

  Problem 1
    - Installation request for drupal/core-recommended ^9 -> satisfiable by drupal/core-recommended[9.1.99].
    - drupal/core-recommended 9.1.99 requires symfony/mime v5.2.0-RC1 -> no matching package found.
  Problem 2
    - Installation request for drupal/core-dev ^9 -> satisfiable by drupal/core-dev[9.1.99].
    - drupal/core-dev 9.1.99 requires symfony/phpunit-bridge ^5.2@rc -> no matching package found.

Which, 🤔

Let's try this? We have symfony/serializer as ^4.4.16 in one place but symfony/mime has 5.2 as a dev dependency according to Packagist. I'm not seeing the result @longwave mentioned of it not being there anymore.

For some reason it rolled back the symfony/mime change...

Oh. symfony/mime is ONLY mentioned in composer/Metapackage/CoreRecommended/composer.json, but not in either composer.json file. Edit: Never mind, I guess this is how it's supposed to be, because it automatically comes back if I remove it.

Leaving symfony/mime and symfony/serializer out of it on purpose for the moment because I'm confused about whether or not indirect dependencies are supposed to be pinned in CoreRecommended and the serializer constraint thing is dicey with letting a production dependency allow a forward major.

Symfony 5.2.0 is out. However I assume this is too late to get into 9.1, but let's see what happens anyway, we will need this for 9.2.

+-------------------------------+--------+--------+
| Production Changes            | From   | To     |
+-------------------------------+--------+--------+
| symfony/mime                  | v5.1.8 | v5.2.0 |
| symfony/var-dumper            | v5.1.8 | v5.2.0 |
| symfony/deprecation-contracts | NEW    | v2.2.0 |
+-------------------------------+--------+--------+

+------------------------+--------+--------+
| Dev Changes            | From   | To     |
+------------------------+--------+--------+
| symfony/phpunit-bridge | v5.1.8 | v5.2.0 |
+------------------------+--------+--------+

I think it's probably too late for 9.1 too, it's a shame we are only a week or two out. However staying on 5.1 we do have an agreement with Symfony that we can help them backport security releases to versions we're using once they're out of official support, so we might get lucky, or if we're unlucky there is a way to deal with it. I would not actually be opposed to jumping up during the release candidate either considering the updates require zero core changes, but between the possibility of a security release in 3+ months or breaking contrib with 9.1.0 who knows...

Note this adds symfony/deprecation-contracts but I don't think we need an independent dependency evaluation for that since it'll be covered by Symfony in general. It contains just one conditionally declared global function, trigger_deprecation().

Given everything is green I don't see a reason to not RTBC this - although leaving the release manager review tag on so that xjm is able to chime in on the above. It definitely makes sense to get this into 9.2.x asap.

So for me the goal of this issue didn't really have to do with 5.2 itself, but with the problem of having 10 months in perpetuity where the minor we're using doesn't have security coverage. We have this issue with a few components for D9, and potentially with all of them for D10.

One step we've taken to make this better with 5.3 is to move the 9.2.0 release date back two weeks into mid-June, so that we can definitely update to the stable minor of Symfony before tagging 9.2.0-rc1.

However, that still doesn't solve the problem of the test failures in #35. If we're not going to solve that here, we need a followup to solve it before 5.3 goes into beta. Something like "Allow betas and release candidates of core to use betas and release candidates of Symfony" or something.

Given that we're releasing 9.1 in probably 24h or so, I don't think we should try to backport this to 9.1.x at this point. We'll have to take the hit and provide security coverage for these various components until Dec. 2021 (which the framework managers coordinate with upstream).

Posted #3186364: Allow pre-release dependencies in Drupal pre-release milestones.

Since this wasn't committed to 9.1, let's make sure to include this in the 9.2 release notes, unless it's been superseded with additional updates since then.