gitlab artifact caching doesn't work due to differences with project ID and build directories

As well as the project ID being different, we're also running into both the yarn lint and phpstan caches using absolute paths.

Tried a rewrite as part of the pipeline process and it partially helps (i.e. phpstan fails to use the cache, but gives less criteria it failed by), but not enough:

https://git.drupalcode.org/issue/drupal-3470641/-/jobs/2585012

https://git.drupalcode.org/issue/drupal-3470641/-/jobs/2585008

Possible workaround mentioned here is to use bind mounts:

https://github.com/phpstan/phpstan/issues/8599#issuecomment-2059227168

e.g. we'd bind mount the current directory to /tmp/build, cd, do the composer install + run the linting jobs there both for the cache warming job and for the individual linting jobs - that way everything runs in the same directory as far as yarn and phpstan are concerned.

However to fully test that we'd need to commit the change to HEAD so that the cache warming job runs with the bind mount, then see what the effect on pipelines is after that. Can probably hack the API URLs to point to the pipeline instead of core just to make sure it doesn't break things, but not easy to demonstrate it actually works.

An extra complexity here is while phpstan gives you an idea why the cache wasn't used, none of the yarn linting tools do (or at least I haven't found it). But probably if we can make phpstan work, the others will work too.

One thing is that in a way this vindicates the idea to use artifacts instead of distributed caching because with distributed caching we'd have a lot less control over how the cache files are generated, so would get cache misses all over the place with even less information on how to debug.

Tried a bind mount but the user in the pipeline doesn't have permission to create one. This might need some help from infrastructure, i.e. can our k8s runner setup allow the bind mount, or can we do something there to have a consistent directory name?

This looks like it should work.

But cant get the cached artifacts to download from the issue fork project even after changing the project id.

I think to test this, it would also be necessary to hack the lint warming job to run on MR pipelines (which it doesn't), so that it creates the artifacts in the first place.

Left one suggestion on the MR to try to explain a bit more what's going on in the comment. Apart from that it looks good. Will be hard to be 100% confident that it's working until after we commit it unfortunately, but that's the nature of what we're fixing.

Committed/pushed to 11.x and cherry-picked to 11.0.x, thanks!

Doesn't cherry-pick cleanly to 10.4.x but I think we might want it there for the next couple of years of backport MRs.

https://git.drupalcode.org/issue/drupal-3421014/-/jobs/2885532 definitely found the cache:

9538/15309 core/modules/system/tests/src/Kernel/Extension/ModuleHandlerTest.php cached
 9539/15309 core/modules/system/tests/src/Kernel/Form/ElementsFieldsetTest.php cached
 9540/15309 core/modules/system/tests/src/Kernel/Form/FileElementTest.php cached
 9541/15309 core/modules/system/tests/src/Kernel/Form/FormElementLabelTest.php cached
 9542/15309 core/modules/system/tests/src/Kernel/Form/FormElementMaxlengthTest.php cached
 9543/15309 core/modules/system/tests/src/Kernel/Form/FormObjectTest.php cached
 9544/15309 core/modules/system/tests/src/Kernel/Form/ProgrammaticTest.php cached

The output is very verbose, but had previously left it like that so we could verify that it's using the cache, which it wasn't until this issue got fixed!