Problem/Motivation

When we did #3549110: [CI] Performance pipeline execution drops warmed caches we hit an immediate snag in that the cache write happens only if all the test jobs pass - that does not happen often due to random fails.

Steps to reproduce

Proposed resolution

Have the lint jobs themselves generate the cache files and then cache them during the test stage.

Remaining tasks

User interface changes

N/a

Introduced terminology

N/a

API changes

N/a

Data model changes

N/a

Release notes snippet

N/a

Issue fork drupal-3558592

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

alexpott created an issue. See original summary.

alexpott opened merge request !13864

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Active » Needs review
mondrake’s picture

mondrake
Location 🇮🇹
commented
Status: Needs review » Reviewed & tested by the community

It's a bit off a job with that name in the 'test' stage, but that's cosmetics. Would it be possible to move it to the 'lint' stage waiting for the other jobs in that stage to complete using needs?

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

@mondrake no. Because then it wouldn't run on performance jobs and we'd be back to the problem caused https://gitlab.com/gitlab-org/gitlab/-/issues/458828

The 'Write lint caches' needs the artifacts from either the individual jobs OR the 'Lint cache download' job. The fix for this is to move away from using artifacts for this and do #3558489: Upload lint cache files to a S3 bucket (or somewhere else)

mondrake’s picture

mondrake
Location 🇮🇹
commented

Thanks

catch closed merge request !13864

catch’s picture

catch
he/him
Primary language English
commented
Version: 11.x-dev » 11.3.x-dev
Status: Reviewed & tested by the community » Fixed

OK let's keep going. Committed/pushed to 11.x and cherry-picked to 11.3.x, thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • catch committed 6b4cc91a on 11.3.x 
    task: #3558592 Improve lint cache warming even more

By: @alexpott
By: @...

  • catch committed 70c84b70 on 11.x 
    task: #3558592 Improve lint cache warming even more

By: @alexpott
By: @...

  • alexpott committed dd92a9c6 on 11.3.x 
    task: #3558592 (follow-up) Improve lint cache warming even more

    By...

  • alexpott committed d0cc7c49 on 11.x 
    task: #3558592 (follow-up) Improve lint cache warming even more

    By...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.