Problem/Motivation

It's currently not possible to run the Repeat Class Test job for WebDriver tests without attaching the selenium chrome service like:

  services:
    - <<: *with-database
    - <<: *with-selenium-chrome

It would also be good to be able to control the number of repeats, currently this is hardcoded to 100.

Steps to reproduce

Try to run the repeat class test pipeline against a FunctionalJavascript test

Proposed resolution

Add the service by default.

Issue fork drupal-3519177

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

acbramley created an issue. See original summary.

acbramley opened merge request !11826

acbramley’s picture

acbramley commented
Status: Active » Needs review
mstrelan’s picture

mstrelan commented

Yes please. While we're at it, can we add a var for number of repeats? It's currently hardcoded at --repeat "100". Would be good if we could set this var via the UI like REPEAT_TEST_CLASS.

We can do a separate issue for that, or we can re-title / re-purpose this issue to include this.

catch’s picture

catch
he/him
Primary language English
commented

Yes! Kept meaning to open an issue for this and hadn't done it yet, and agreed a variable for number of tests would be good - both for really long running tests that can time out with 100 runs, and also when you need 500, I think it's fine either here or split out.

acbramley’s picture

acbramley commented
Title: Attach selenium service in Repeat test class job » Attach selenium service in Repeat test class job and add variable for number of repeats
Issue summary: View changes
Status: Needs review » Needs work

Would be good if we could set this var via the UI

You can do it via the UI? 😲

Happy to add the variable here given this is such a small change.

catch’s picture

catch
he/him
Primary language English
commented

You can do it via the UI?

Once you have a pipeline running, you can manually run the job, and gitlab allows you to set environment variables there - this currently supports only the class name, but if we do the same for the number of runs, then it'll just be setting two variables - would be nice to keep it defaulting to 100 though I think so it's not required.

mstrelan’s picture

mstrelan commented

For example, go to https://git.drupalcode.org/issue/drupal-3519177/-/pipelines/473447, click on Repeat Test Class then enter the variables.

acbramley’s picture

acbramley commented
Status: Needs work » Needs review

Just found the help text in the job about how to set the variables in the UI. That's awesome.

Added REPEAT_COUNT

acbramley’s picture

acbramley commented

Tested with both variables here https://git.drupalcode.org/issue/drupal-3519177/-/jobs/4965595

mstrelan’s picture

mstrelan commented
Status: Needs review » Reviewed & tested by the community
catch’s picture

catch
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Fixed

This is great, let's do it. Normally I'd leave an issue open a bit longer, but it's only changing a manually triggered test job, so quicker to commit than leave this then come back to it a day later.

  • catch committed 80223725 on 11.x 
    Issue #3519177 by acbramley, mstrelan: Attach selenium service in Repeat...

catch closed merge request !11826

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

xjm’s picture

xjm
she/her
Primary language English
commented
Status: Closed (fixed) » Needs work
Issue tags: +Needs change record, +Needs documentation updates

Sorry for issue necromancy but it would be nice for more people to know about this. :)

mstrelan’s picture

mstrelan commented

Can you be more specific about which documentation to update? There is https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... and https://www.drupal.org/about/core/policies/core-change-policies/how-to-d... but neither of these mention REPEAT_TEST_CLASS. Since this issue is about a minor enhancement to #3393475: Enable running the same test multiple times on gitlab ci for random test failures I suspect that is the issue that should have added documentation. Probably better to raise a new issue for it at this point.