Problem/Motivation

Occasionally, something goes wrong with DrupalCI infrastructure, and patches that are otherwise fine fail with "CI error". While it makes sense for these errors to not be counted as passing test runs, it does not make sense for them to cause retested RTBCs to be marked NW, as it is extremely unlikely a DrupalCI error could actually be caused by a previously testing patch.

Proposed resolution

Instead, the issues should remain at RTBC when they have a CI error instead of an actual failure, and be retested as appropriate.

Comments

xjm created an issue. See original summary.

Wim Leers’s picture

+1! Dozens of times I've had to re-test patches for this. If this would be possible, this would really help :)

jonathan1055’s picture

Yes I agree this would be great. There is a current problem on 7.x patch testing, which has been failing with 'CI error' since 15 Dec #2837015: CI problem with permission when testing 7.x patches. Having issues not revert to NW would be good in this situation.

Mixologic’s picture

Project: DrupalCI: Drupal.org Testing Infrastructure » Project issue file test
Version: » 7.x-3.x-dev

Work is underway to mostly eliminate CI error from happening at all and be more explicit about what those error conditions are, so many of these wont happen again, but they will still be test fails (patch doesnt apply, @group missing, etc)

The most recent bout of CI errors was due to the new deployment changing all of the signaling to reflect a testing application (0 = pass, 1 = failed tests, 2 = exceptions during testing and > 2 = abnormal error state).

The problem was that Jenkins saw anything > 0 as a FAILURE, and drupal.org is currently interpreting Failure to mean CI error, when it was really just some failed tests, or failed patches.

So, anyhow, we're working on it, but ideally CI errors become less common, and we'll have to address what *types* of test states should/should not mark an issue as needs work.

Wim Leers’s picture

Happening again now.

drumm’s picture

Title: DrupalCI errors should not mark RTBC patches NW » CI errors should not mark patches NW
Assigned: Unassigned » drumm

  • drumm committed c430a44 on 7.x-3.x
    Issue #2836371: Replace CI error with re-queue
    
drumm’s picture

Status: Active » Fixed
Issue tags: +needs drupal.org deployment
Wim Leers’s picture

OMG OMG OMG OMG!

Thank you!

This will save us so much pain!

drumm’s picture

Issue tags: -needs drupal.org deployment

This has been deployed.

Wim Leers’s picture

<3 <3 <3

  • drumm committed fad4d9c on 7.x-3.x
    Issue #2836371: Only re-queue CI error when job failed
    

  • drumm committed ff367f8 on 7.x-3.x
    Issue #2836371: Fix last commit
    

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.