The composer.json in Lightning is missing the extra.enable-patching key. This prevents child distributions that are referencing lightning as a dependency to get the patches being applied by lightning when doing a composer-update. If you are doing an initial build it seems to work, but when updating versions in the child's composer.json the patched modules (like panelizer, metatag, etc) get removed but then not patched again.

This can sometimes work ok if you haven't cleared the composer-cache. Because it caches the patched version of the modules. But there is a bug in composer-patch (https://github.com/cweagans/composer-patches/issues/66) that fails to apply an updated patch because the previous patch is cached. This has caused us to start using composer clear-cache more and more, and when the composer cache is cleared, the dependency patches in Lightning composer.json are not applied.

Adam created a test branch 8.x-1.x--add-patches-key that adds the "enable-patching" key and that solved the problem for us here. So I suggest that branch get merged into master.

Comments

mpotter created an issue. See original summary.

balsama’s picture

balsama
he/him
Primary language English
Location boston
CreditAttribution: balsama at Acquia commented

Hey Mike. I'm not 100% convinced that the addition of the "enable-patches" key is _really_ what fixed the issue. That's definitely not what that key is for. I suspect something else is happening.

From the composer-patches documentation:

If you want your project to accept patches from dependencies, you must have [enable-patches: true] in your composer file

So the intention of the flag is specifically to allow dependencies to define patches. It doesn't shouldn't take into consideration whether or not the flag is set to true in the dependency itself. I also reviewed the commit that added that flag (and consulted with the author) and didn't find anything that made me think it is.

For the record, I've definitely seen this occur, but it's not clear to me what the steps are to reproduce. Wiping out the local repo seems to fix it. I wonder if switching from the packagist to Lightning's VCS on GitHub (which was needed to get the test branch) would have resolved the problem by itself. In other words, we introduced two changes when testing. We:

  1. Changed source of lightning (from packagist.drupal-composer.org to github.com)
  2. Added `allow-patchess: true` to Lightning's composer.json file

I suspect (or at least want to test) that just #1 would have fixed the problem. In order to test, we would need another project in that weird "won't accept patches" state. And then we would need to point that project's root composer.json to the GH (VCS) repo of lightning rather than the packagist hosted one.

Anyway, I'm happy to set that flag to true in Lightning - even if the intended purpose wasn't to allow/prevent stuff like this. But I really want some more data that indicates it's related first. @mpotter - do you have to steps to reproduce the state where a root composer.json that depends on Lightning doesn't apply patches defined in Lightning? (Again, for the record, I've definitely seen that situation, but it's not clear to me what conditions cause it)

mpotter’s picture

mpotter CreditAttribution: mpotter at Phase2 commented

Yes, you might be correct about the change from packagist.drupal-composer.org to github fixing this.

I reproduced this on a fresh site by downloading our drupal.org/project/octane distro and then running composer-install. None of the Lightning patches were applied.

In this case we are pulling in Lightning as a dependency, which makes me wonder if the enable-patching is working at all if it's supposed to be the flag that allows dependencies to patch. Could there be a case where even though this flag is set in the child distro that it still doesn't apply patches from dependencies?

I will try some more test cases today to pin this down. Of course, the frustration is that every one of these tests takes about 30 minutes with the slowness of composer (which is a whole different frustration of mine). Our other projects have now just gone through and applied patches manually to get back up and running.

balsama’s picture

balsama
he/him
Primary language English
Location boston
CreditAttribution: balsama at Acquia commented

I reproduced this on a fresh site by downloading our drupal.org/project/octane distro and then running composer-install.

Great. I'll see if I can reproduce this today too.

balsama’s picture

balsama
he/him
Primary language English
Location boston
CreditAttribution: balsama at Acquia commented

Rereading your comment:

I reproduced this on a fresh site by downloading our drupal.org/project/octane distro and then running composer-install.

That implies you're downloading the D.O packaged tarball (or using drush to download it). That packaged version of Octane will have the automatically generated composer.json at the root. That file doesn't even have Lightning as a dependency.

If I clone Octane (the unpackaged version from here: https://git.drupal.org/project/octane.git) and run composer install I get Lightning and all of its patches as expected.

mpotter’s picture

mpotter CreditAttribution: mpotter at Phase2 commented
Status: Active » Postponed

No, I was downloading the git and running "composer install". But just did that again clean this morning and it worked fine. So this is very perplexing. After further testing it seems that deleting the entire vendor directory is what fixes it, so it's possible the site I was testing yesterday wan't actually clean and had a previous version in the /vendor directory.

I'm tearing my hair out trying to reproduce this now. Yesterday is was this huge problem and today it seems fine. And testing is just too slow to really pin down the problem. Until I can get composer working faster I probably can't do more to figure this out.

Help would be welcome from anybody that has time. It's clear that "something happens" to cause this and the solution is to "clean build from scratch". But since that is so slow and it only triggers sometimes, I'm struggling trying to come up with an upgrade workflow for our devs that is always going to work and which isn't so slow that our devs will actually use it.

Right now we have projects giving up and just using their own composer.json (copy/pasting from Lightning) just so they don't have any upstream profile dependency. But that defeats the purpose of inherited profiles and means they are going to get off track when it comes to keeping their Lightning dependencies and patches updated themselves. Obviously that's not ideal.

Going to "Postpone" this one until I have more time to work on it.

dwkitchen’s picture

dwkitchen CreditAttribution: dwkitchen at Johnson & Johnson commented

I can reproduce similar problems and I think the issue is in composer-patches.

In my test example the patches weren't being applied, so I deleted vendor, web and composer.lock and ran composer install, all work correctly and installed with patches.

However, I then updated and added a new patch to the profile. When I ran composer update in the site it updated the profile but didn't apply the new patch to drupal/core. To get the patch to apply I had to delete the composer.lock file and then run composer update.

balsama’s picture

balsama
he/him
Primary language English
Location boston
CreditAttribution: balsama at Acquia commented

cweagan's released v1.6 of copmoser-patches which should permanantly resolve these issues. Can you update to 1.6 and see if that resolves your issues?

See https://github.com/cweagans/composer-patches/issues/26 and https://github.com/cweagans/composer-patches/pull/30 among others.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Postponed » Closed (outdated)

This has not seen any activity in six months, so I think it's time to close this out.

Ralf Eisler’s picture

Ralf Eisler CreditAttribution: Ralf Eisler commented

In the last days I tried different updates and fresh installs of Drupal Lightning with patch-errors with different snapshots of Composer.

For example:

PHP 7.1

Composer version 1.6-dev (edece864e7e4c668dcad6601df70777882d22116)
composer create-project acquia/lightning-project:^8.1.0 my.project --no-interaction

Errors (2 examples):

- Applying patches for drupal/core
    https://www.drupal.org/files/issues/2652138-41.patch (2652138 - ImageFormatter does not support SVGs)
   Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2652138-41.patch

  - Applying patches for drupal/workbench_moderation
    https://www.drupal.org/files/issues/2668006-2.patch (2668006 - Provide logical access control for when IPE should be applied to moderated nodes)
   Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2668006-2.patch