Consider using Symfony flex to allow switching between major Symfony versions

I committed #3020301: Remove composer integration PHP version test (rely on DrupalCI PHP version runs instead), so this is no longer postponed.

If we commit this, we'd need to be extra careful updating dependencies in composer.lock

I don't suppose Composer has any support for indicating preferred version, does it? E.g., "prefer lowest major, but highest minor"? Or "prefer the first version range in a list that is separated by |"?

How about this as a way of addressing #19?

If we commit this

If we don't know yet if we want to commit this once it's ready, why is this issue prioritized as Critical?

I think we want to commit this and release with 8.7 if at all possible so people can turn up more problems that we can fix in 8.8 and then release 8.8 with optional support for Symfony 5 so people can turn up more problems that we can fix in 8.9 and release Drupal 9 at the same time with 8.9 requiring Symfony 4 or 5 based on how far we get.

I'm a bit worried about the impact this will have on all the drupal-project sites out there if we commit it. Even if we add it, the php version thing won't do anything for them, because they don't use the root-level composer.json. And I'm actually kinda surprised that passed, because shouldn't that prevent the new php7-only phpunit version from being installed on testbot?

Unfortunately it is quite hard to test what would happen because you can't change composer dependencies with patches. If we do decide to commit it then I'll try the dev branch on our install profile, to check how well it is going to work.

The PHP version trick will force us to stay on Symfony 3 for now, but it seems a bit fragile if another dependency raises its PHP dependency to 7.1.3 and we want to update to that in a minor (once we require PHP 7.1 of course)

I think we should keep Drupal 8 on PHP 7.0 for the rest of its life. Ubuntu 16.04 has PHP 7.0 and is supported until April 2021. Debian 9 has PHP 7.0 and has LTS until June 2022. Drupal 9.0, of course, can require a higher PHP version.

I'm actually kinda surprised that passed, because shouldn't that prevent the new php7-only phpunit version from being installed on testbot?

The patch adds --ignore-platform-reqs to the drupal-phpunit-upgrade command.

I'm a bit worried about the impact this will have on all the drupal-project sites out there if we commit it. Even if we add it, the php version thing won't do anything for them, because they don't use the root-level composer.json.

Aren't those exactly the projects that we want updating to Symfony 4 and reporting issues to us? If the project uses modules/packages that are not compatible with Symfony 4, then isn't it the responsibility of those incompatible modules/packages to specify their requirements in their composer.json file? And per #21, a drupal-project site can add those requirements to their root composer.json too. But you're right that it's a potential BC break for sites that are incompatible with Symfony 4 but don't have that reflected in any composer.json file.

There's also --prefer-lowest which is fine for our Symfony range because 3.4 is the lowest minor we specify, although we'd need to start specifying patch updates then for things we rely on.

Would that require us to update the minimum patch number every time any vendor library has a security update? Maybe that's fine, since if we're updating our composer.lock file anyway, then why not update composer.json at the same time?

Projects that depend on a Symfony-4-compatible drupal/core package could add "symfony/lts": "^3" to their dependencies if they don't want to upgrade to Symfony 4 just yet. That package simply adds a conflict for version 4 of each Symfony package. It is marked as abandoned at Packagist though. However, since the list of Symfony 3 packages won't change anymore, it should be save to use it for this purpose.

Echoing berdir's #24. This feels extremely unwise.

The documented starting point for every Drupal Commerce site is "composer create-project".
This will silently switch a large portion of our users to Symfony 4. That makes it the equivalent of Drupal core switching the tarball to Symfony 4.

It is worse to have some people on Symfony 4 (and experiencing contrib incompatibilities) and some on Symfony 3, than to just move everyone to Symfony 4.

I agree with @Berdir and @bojanz - this feels very risky.

Also,

+++ b/composer.json
@@ -14,7 +14,10 @@
+            "php": "5.5.9"

This is tricky. We've tried similar stuff like this before. But this risks breaking tar installed / git installed sites that are on PHP 7 and have other non-drupal dependencies that are PHP 7 only.

Let's return to what @catch said in #25:

we want to commit this so that we can eventually test contributed projects (and core patches) with minimum and maximum versions of dependencies, and also to make Symfony 4.3, 4.4, 5.0 and 5.1 compatibility easier to track. This is the only way to allow contributed branches to test for Symfony 4 compatibility issues prior to the 9.x branch opening.

Whether we can commit this exact patch in its current form or not is a different question.

(Emphasis mine). Can we make a different change that would not have the same side effects of forcing anyone to Symfony 4, yet would still allow us to gather feedback on Symfony 4 support as well as let contribs adopt optional Symfony 4 support?

Today we discussed when and whether to commit this change so that people can test it. Options discussed:

1. Commit to 8.7.x in time for beta.
2. Create an 8.7.x-symfony4 feature branch (which would not be updated by committers after branching or tested on infrastructure)
3. Branch 9.x within the next month and commit it there
4. Commit this to 8.8.x when it opens and get feedback/testing there, with the possibility of reverting it if it's too disruptive.

The committer team discussed this and we mostly think option #1 is too risky. Option #4 is what we agreed on. So we can commit this to 8.8.x when we branch it, which will be in a month or so, and then do a communication push around testing it and giving feedback.

Edit: The above isn't signoff on the current implementation, just what we would do if we were to commit the current implementation or something like it.

Re #28 / #33, https://github.com/symfony/lts says that it's abandoned and recommends using symfony/flex instead. Here's a patch that does that. It therefore removes the #31 hunk as well.

Symfony/flex has a composer.json dependency on PHP 7.0, but if we're not planning to commit this until the Drupal 8.8.x branch per #34, then that's not a problem per the proposed resolution of #2842431: [policy] Remove PHP 5.5, 5.6 support in Drupal 8.7:

The 8.8.x branch will not be tested against PHP 5.

This patch adds the "symfony": { "require": "^3" } configuration (used by symfony/flex) to the root composer.json (not core/composer.json). This would set the default for sites that use Drupal's root composer.json. Sites that use their own root composer.json could change that requirement to allow for Symfony 4 if they want to help us test it. How does that sound?

Sorry, the interdiff in #35 is incorrect. The patch is correct (but badly named, because it's in no way the same as the one in #20).

Here is the correct interdiff between the patch in #20 and the patch in #35.

#2976394: Allow Symfony 4.4 to be installed in Drupal 8 reached a point where 8.8-dev is now Symfony 4 compatible (other than test fails for deprecations for Symfony 5). So we are back here to figure out how to support this.

Can you be more specific what needs to be figured out?

Reading through the comments it seems the decision was made already.

4. Commit this to 8.8.x when it opens and get feedback/testing there, with the possibility of reverting it if it's too disruptive.

If it will have to be reverted, then it should be before 8.8 alpha, so 4 months left for testing SF4 on D8.

Or do you think

3. Branch 9.x within the next month and commit it there

is a much more logical/cleaner way forward now?

Sites that use their own root composer.json could change that requirement to allow for Symfony 4 if they want to help us test it. How does that sound?

So that means any site that has been built using the recommended way to build Drupal sites will use Symfony 4 the next time they run composer update - the official user guide docs are here https://www.drupal.org/docs/user_guide/en/install-composer.html and drupal-composer/drupal-project uses drupal/core not drupal/drupal.

So that means any site that has been built using the recommended way to build Drupal sites will use Symfony 4 the next time they run composer update

In general, how do composer project templates release bug fixes or other improvements to their existing users? For example, if we also added something like #35 to drupal-composer/drupal-project, is there anything within existing conventions of how people use composer after creating a project from a template that would notify them that an update to the template exists?

I think the project template is exactly that - a template that is used once to template your project - there's no link back to the template as far as I know.

Yeah if you want new things from the template you need to copy them.
Doesn't happen automagically.

But that's fine.

But that's fine.

@aspilicious in this case I'm not sure it is. If I have a drupal project created site and I suddenly get Symfony 4 because drupal/core has relaxed it's version requirements I'm going to be mightly surprised because uptil this point the version of Symfony I'm on has been controlled by drupal/core - I've not had to make this consideration myself. This especially problematic for contrib and custom modules because at the moment they make assumptions about inherited dependencies. And to compound this we don't do deprecation testing by default for contrib so if they are using deprecated code in Symfony 3 they have to do work to find this out.

I, for one, am OK with this potential issue with drupal-composer/drupal-project. Post a CR about symfony 4. Add it to release notes. Then let folks test their sites and possibly add a pin on symfony 3 in their composer.json. All another day in the moving target of being a drupal developer. If it breaks, usually things break spectacularly. Which is easy to catch in even rudimentary testing.

I, for one, am OK with this potential issue with drupal-composer/drupal-project.

I'm OK with this too. All my projects have pinned to Symfony ^3 for some reason or another. Folks who dont do that will see a failure from Contrib/Custom and can take appropriate action. Minor core upgrades always require some work.

I, for one, am OK with this potential issue with drupal-composer/drupal-project.

So can I do the Symfony update to 4 if I don't use this components because I converted my websites this way ?

This is not going to happen due to various concerns raised here. Also we could only do this now in Drupal 8.9 and that is supposed to release alongside Drupal 9, so no point anymore. See you in #3088369: Update Drupal 9 to Symfony 4.4-dev though!

We still have the same problem the fact we ship a lock file and don't test against our minimum and maximum dependencies. It feels' like all that has to be fixed before we do this. Also we know we're not compatible with SF5 because we're currently skipping some SF4 deprecations.

The beta clock has just about run out (and honestly I think it was beta deadline anyway).

We might want to revisit this during D10 development, since there's a chance we'll be trying to bridge our way from SF4 to SF6.

@catch linked to this issue in Slack and asked for reviews, so here are some thoughts:

As I understand it this will effectively require downstream users (both contrib and end user sites) to declare which version(s) of Symfony they support, instead of relying on Drupal core's major version to infer this. To me this brings us back to the old question of is Drupal a product, or a framework? If it's a product, then I think we should lock to a Symfony version and be done with it; but if it's a framework then I see the advantages of allowing flexible Symfony versions. But in many ways Drupal is both, and I believe some (product) users will not like the additional load here of having to select a Symfony version to use.

Also, does this risk further fragmentation in contrib? We already have the issue that when we release a new major, contrib has to catch up. But now if we're also asking contrib to support multiple Symfony versions, we risk further incompatibility issues - Drupal core might support two versions of Symfony, but if module A only supports SF5 and module B only supports SF6, that means they can no longer be used together.

Finally I think this was already alluded to above, but we surely need test infrastructure improvements to be able to do this. Once we declare that we support more than one Symfony version, all patches will have to be tested on both versions prior to commit - at least the latest version of each major, if not full min/max testing across all supported majors - to ensure that we haven't broken anything.

Also, do we even need Symfony Flex? Can we ship Drupal 10 with ^5.4|^6.0 dependencies on Symfony, with the lockfile set to Symfony 6? This means people get SF6 by default but can downgrade if they want to.

We've shown in #3161889: [META] Symfony 6 compatibility that we can already support ^4.4|^5.4 in 9.3.x - albeit with some deprecations for Symfony 6.x, but nothing actually broken. Could we support ^5.4|^6.4 to Drupal 10 EOL, and then go to ^7.0 in Drupal 11.0?

@longwave versions still makes sense as core provides independent components (each provides own constraints)

Flex got changes https://symfony.com/blog/symfony-flex-is-going-serverless

As Symfony 5.3 there's https://symfony.com/doc/current/components/runtime.html which is now moving to separate project

It could help to manage core's variations (IIRC there's 3 Kernels already)

Wanted to bump 1 more time as there are a lot of people on this one.