Problem/Motivation

While we want the first stage of tests to help keep core consistent and catch issues with phpstan. There are situations such as architectural issues you would want full tests to run even if phpcs is failing for example.

I would like to suggest that if an MR is in draft then the first stage of tests get a new rule that allows failure.
We would need to ensure that code is not committed from a draft MR as well.

Steps to reproduce

See:
https://www.drupal.org/project/drupal/issues/3525331
https://www.drupal.org/project/drupal/issues/3526388

Neither of those are really in a position to be even in needs review, but it would be valuable to see a full test run even if there is a spelling mistake. This would have the potential to increase resources used by the DA, but we could be cognizant of when an issue uses Draft MRs

Proposed resolution

Rule to allow failure for most jobs in the first column when the MR is in Draft

Remaining tasks

Determine if there is a better way to do this.

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3526516

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

nicxvan created an issue. See original summary.

mstrelan’s picture

mstrelan commented

Could possibly make use of labels on the MR as well

nicxvan’s picture

nicxvan commented

I was concerned about how visible those would be for core committers / reviewers, but that might be a viable way too.

nicxvan’s picture

nicxvan commented

https://www.drupal.org/project/drupal/issues/3515704 may have some examples to pull from.

nicxvan’s picture

nicxvan commented

Changes from gitlab ci here: https://git.drupalcode.org/project/drupal/-/merge_requests/11841/diffs

More notes from @mstrelan:
https://forum.gitlab.com/t/how-to-share-artifacts-even-on-job-failure/69393
https://docs.gitlab.com/ci/yaml/#artifactswhen

So we might add this:
allow_failure: true
artifacts when:always

When a certain label is applied or it's a draft.

quietone’s picture

quietone commented

It would also be helpful to have a way to only run the linting jobs but not the tests. That would be handy for documentation only issues.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented

There is a CI_MERGE_REQUEST_DRAFT variable we could use for this.

When issues are on GitLab then we can perhaps do more with labels to run less tests in certain situations.

longwave opened merge request !14725

longwave opened merge request !14726

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented

index.php is not checked by PHPCS, PHPStan, or CSpell apparently.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Status: Active » Needs work

PHPCS is responsible for generating the vendor artifact, and we can force those to upload even on failure, but https://gitlab.com/gitlab-org/gitlab/-/issues/367229 then prevents the downstream jobs from pulling artifacts from the failed job.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Status: Needs work » Postponed
Related issues: +#3572572: [ci] Cache vendor so we hit Packagist and GitHub less often

As far as I can see the only way to allow PHPCS to fail is to move composer install to its own job.

#3572572: [ci] Cache vendor so we hit Packagist and GitHub less often might be a possible workaround - we can use the vendor cache instead - postponing on that.