Problem/Motivation

Yarn caching is not working for cspell and should be extended to other jobs.

Steps to reproduce

Proposed resolution

Use cache for all yarn installs in lint jobs.
Remove yarn cache - it appears to be handled at a lower level (details to follow).

Remaining tasks

User interface changes

N/a

Introduced terminology

N/a

API changes

N/a

Data model changes

N/a

Release notes snippet

N/a

Issue fork drupal-3558704

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

alexpott created an issue. See original summary.

alexpott opened merge request !13868

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Active » Needs review

Actually it looks as if the whole cache thing is pointless - we already seem to be caching yarn installs at a lower level... need to confirm with da infra but I think we should just remove the cache and install from the CI_PROJECT_DIR/core directory and everything is quick.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

Compare...

No caching in gitlab-ci.yml.... https://git.drupalcode.org/issue/drupal-3558704/-/jobs/7352532

➤ YN0000: · Yarn 4.9.1
➤ YN0000: ┌ Resolution step
section_start:1763549298:resolution_step[collapsed=true]
[0KResolution step
section_end:1763549298:resolution_step
[0K➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
section_start:1763549298:fetch_step[collapsed=true]
[0KFetch step
➤ YN0013: │ 1078 packages were added to the project (+ 277.49 MiB).
section_end:1763549303:fetch_step
[0K➤ YN0000: └ Completed in 5s 280ms
➤ YN0000: ┌ Link step
section_start:1763549303:link_step[collapsed=true]
[0KLink step
➤ YN0007: │ @nightwatch/nightwatch-inspector@npm:1.0.1 must be built because it never has been before or the last one failed
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT > @nightwatch/nightwatch-inspector@1.0.1 build
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT > node preprocessExtension.js
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 📂 Moving to src folder ...
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 🚀 Creating .crx file ...
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT ✅ .crx file created successfully!
section_end:1763549316:link_step
[0K➤ YN0000: └ Completed in 12s 176ms
➤ YN0000: · Done in 17s 799ms

Caching in gitlab-ci.yml.... https://git.drupalcode.org/issue/drupal-3558704/-/jobs/7352479

➤ YN0000: · Yarn 4.9.1
➤ YN0000: ┌ Resolution step
section_start:1763548997:resolution_step[collapsed=true]
[0KResolution step
section_end:1763548997:resolution_step
[0K➤ YN0000: └ Completed in 0s 296ms
➤ YN0000: ┌ Fetch step
section_start:1763548998:fetch_step[collapsed=true]
[0KFetch step
➤ YN0013: │ 1078 packages were added to the project (+ 277.49 MiB).
section_end:1763549023:fetch_step
[0K➤ YN0000: └ Completed in 25s 876ms
➤ YN0000: ┌ Link step
section_start:1763549023:link_step[collapsed=true]
[0KLink step
➤ YN0007: │ @nightwatch/nightwatch-inspector@npm:1.0.1 must be built because it never has been before or the last one failed
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT > @nightwatch/nightwatch-inspector@1.0.1 build
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT > node preprocessExtension.js
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 📂 Moving to src folder ...
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT 🚀 Creating .crx file ...
➤ YN0000: │ @nightwatch/nightwatch-inspector@npm:1.0.1 STDOUT ✅ .crx file created successfully!
section_end:1763549031:link_step
[0K➤ YN0000: └ Completed in 7s 272ms
➤ YN0000: · Done in 33s 787ms
alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Issue summary: View changes
alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

@longwave pointed out at the same time that I realised we've not caching the correct thing! See https://gitlab.com/gitlab-org/gitlab/-/issues/364754

Fixed MR to cache yarn's cache...

If yarn installs using the cache we will not see

YN0013: │ 1078 packages were added to the project (+ 277.49 MiB).

in the logs.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

Yeah the latest MR is correctly caching yarn install...

Compare https://git.drupalcode.org/issue/drupal-3558704/-/jobs/7353352#L38 (properly cached) with https://git.drupalcode.org/issue/drupal-3558704/-/jobs/7353271#L39 (not properly cached)

fjgarlin’s picture

fjgarlin commented
Status: Needs review » Reviewed & tested by the community

It looks good!

Uncached:

➤ YN0013: │ 1078 packages were added to the project (+ 277.49 MiB).
➤ YN0000: └ Completed in 6s 416ms

Cached:

➤ YN0000: └ Completed in 1s 365ms

The code looks good too.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Reviewed & tested by the community » Needs work

@longwave pointed out ...

longwave
  6 minutes ago
@alexpott instead of YARN_GLOBAL_FOLDER inline what about setting YARN_ENABLE_GLOBAL_CACHE=false globally
alexpott
  6 minutes ago
Then we have to set YARN_CACHE_FOLDER… seemed simpler to do with 1 variable
longwave
  6 minutes ago
YARN_CACHE_FOLDER defaults to ./.yarn/cache anyway
longwave
  6 minutes ago
the point of disabling global cache is that you get a local one instead
alexpott
  5 minutes ago
Sure why not then - guess it is more logical
longwave
  3 minutes ago
lets also set YARN_ENABLE_MIRROR=false as that seems to provide some more speed up (otherwise it writes the global cache anyway) https://yarnpkg.com/configuration/yarnrc#enableMirror
longwave
  3 minutes ago
https://github.com/specklesystems/speckle-server/pull/4651
longwave
  3 minutes ago
unsure how yarn detects if it is running in CI but apparently it does
alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Needs work » Needs review

  • longwave committed f548a439 on 11.3.x 
    ci: #3558704 [CI] Fix yarn caching on lint jobs

By: @alexpott
(cherry...

  • longwave committed 47fd1ee0 on 11.x 
    ci: #3558704 [CI] Fix yarn caching on lint jobs

By: @alexpott
longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Version: 11.x-dev » 11.3.x-dev
Status: Needs review » Fixed

Committed this from NR when I thought it was RTBC after having it open in two different tabs - oops. Looks correct to me though, we can fix quickly in followups if there are issues.

Committed and pushed 47fd1ee08db to 11.x and f548a4396ef to 11.3.x. Thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

longwave closed merge request !13868

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.