How to deal with a random test failure

A patch/MR may fail in a test unrelated to the changes. This may be because of a know random test failure. The instructions here will help to determine if the failure is a know random failure. These most often happen with JavaScript tests, but not always.

1. Confirm the failure is a random failure.
   1. You can see why the test failed by clicking the link for the test results. The link text will looks something like: PHP 5.5 & MySQL 5.5 18,565 pass, 1 fail.
   2. Check the Drupal core test results page for other identical failing test results.
      • Click on any failed test links and see if they have the same result as the failure on your issue.
      • If the same test fails more than once in a row for one or more environments, HEAD might be broken. Search for an existing issue related to the failure, or ask about it in Drupal Slack.
   3. Read the test result output carefully.
      • Is the test that's failing being added in the issue? If so, it is not an existing random test failure, and should be fixed in the issue.
      • Is the test that's failing test coverage related to the changes in the issue?
      • Pay special attention to JavaScript test failures that might be related to the changes in the issue. This could indicate the issue is introducing a new random test failure, and should not be committed until it is resolved. If you believe this to be the case, seek feedback from the Drupal core committers or Drupal Association infrastructure team on whether the test failure has happened before and on how it should be fixed.
   4. Requeue the test job using the "Add test/retest" link underneath the patch or MR. If the test fails a second time with the same exact result, it's not likely to be a random test failure.
2. Look for an existing issue for the test failure. If you find one, comment on that issue, linking to your issue.. An existing issue can be found using the following steps.
   1. Search issues with the Random test failure issue tag.
   2. Check the meta issue for reports of the failure. There may be a child issue already made but the issue tag was forgotten. #2829040: [meta] Known intermittent, random, and environment-specific test failures
3. If you don't find any existing issue for that random failure.
   1. Create a new separate issue.
   2. Add a link to the test results that has the failure. The link will look something like: https://www.drupal.org/pift-ci-job/286183
   3. Add the tag, "Random test failure".
   4. Add the issue to #2829040: [meta] Known intermittent, random, and environment-specific test failures.