Add all modules & themes to the composer replace section

Themes should be included too.

What do we do with the already deprecated modules? Do they need to be in branches replace still?

Those should be in replace too. Any module core provides it is replacing.

Do we backport anything to 11.3?

Yes, every supported branch should have an updated list.

I recommend adding a test to ensure the replace list matches what modules & themes are included in core at the time, so it does not get outdated.

The one concern I have with adding every module is modules like e.g. contact module where they already have users in contrib, because that module has been available for a while - contact module has nearly 1000 users and is compatible with >= 11.3. So if we add a replace in 11.5/4/3 those users will be shifted back to the core version, and won't have a way to install the contrib version until 12.0.0 comes out. I don't know whether it's even possible to undo a replace in composer.json, whereas it's possible to add one to project composer.json

While it's unlikely at the moment, it would also be possible for contrib versions to have changes which prevent moving back to the core version.

One option would be to only add the replace for modules to versions where the contrib module is not compatible with that version. So given telephone is compatible with Drupal >= 11.4, we'd only add the replace to 10.6 and 11.3 https://www.drupal.org/project/telephone - and then 11.4 and above would get the contrib version unless they add their own replace (which they can). But I think this could use more opinions.

Also bumping this to critical because it's preventing composer updates for some sites.

One other option may be to follow the Composer documentation suggestion and restrict the replace to self.version.

At the project level, users would need to provide the version constraint on the require, for example, composer require drupal/contact:^1.0, to get the contrib version specifically, so that the core replace does not apply.

I don't see the Olivero and Claro in the MRs.

For history, the replace section, using "self.version" was added in #2456009: Add a "replace" section to core/composer.json because the dependency was not resolved. Then, it was removed in #3292380: Remove the "replace" section from core/composer.json because it "explicitly prevented composer from downloading contributed versions of core modules". Now we are adding it back because sites are getting the contrib version which is not compatible with their version of Drupal.

Returning to 'self.version' seems like a good approach. Then, sites have control of the version through their composer configuration.

And I don't see that any docs need to be updated.

Not sure how the Priority changed.

I was thinking that we need a test to make sure the the replace section has the correct data for each branch. And it turns out that #3292380: Remove the "replace" section from core/composer.json removed such a test. So, I restored that and modified it to include themes as well.

We still need a decision on #21

It'd be helpful for the IS to explain why this is essentially reverting a change we made in D9. Why is the scaffolding code not sufficient anymore? Would we do away with some of the scaffolding code if we fixed this? How did it start preventing Composer updates?

It might also be good for the list to be auto-generated by something in composer/ so that we don't end up with mismatches.

Update issue summary

Would it be possible to generate this replace list from the core modules/themes rather than maintaining the list manually in core/composer.json?

Since the new test already discovers the non-hidden core extensions dynamically, I'm wondering if we could use the same source of truth when generating the Composer replace entries. That would help avoid the list getting out of sync when modules or themes are added or removed.

@nitinkumar_7 it's a probably possible in theory but we don't have any tooling around it so would be a follow-up issue if at all.

Why is the scaffolding code not sufficient anymore? Would we do away with some of the scaffolding code if we fixed this? How did it start preventing Composer updates?

I wish to reiterate from previous comments I’ve made.

This was broken even in D9, perhaps not as severely as it is now ( older core releases had the replace line and thus the changes never impacted them) however the warning signs were there in #3292380: Remove the "replace" section from core/composer.json that this would become a life long problem. We had 4 years to work it and chose not to.

The community provided a fair amount of support in the #support Slack channel to deal with that being a core BC break at the time as it felt we had no chance to see the removals undone due to core desire to test the next release.

IMO https://git.drupalcode.org/project/project_composer/-/work_items/3582324 would help on the composer side, at least it would prevent cases where contrib requires the the core module from forcing a download from contrib until they test and add the dependency.

More fundamentally:
For this to ever really work well, it likely needs a diffrent composer package name, or the contrib modules need to make releases for D8+ so that require lines match (these would most likely be simple releases where release X.Y.Z exists only to require a dependency upon core of equal and same version).

Side item: core really should consider a sysadmin gate, someone who understands the system admin side impacts of these type of changes who can guide the core devs.