Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Problem/Motivation
Occasionally, something goes wrong with DrupalCI infrastructure, and patches that are otherwise fine fail with "CI error". While it makes sense for these errors to not be counted as passing test runs, it does not make sense for them to cause retested RTBCs to be marked NW, as it is extremely unlikely a DrupalCI error could actually be caused by a previously testing patch.
Proposed resolution
Instead, the issues should remain at RTBC when they have a CI error instead of an actual failure, and be retested as appropriate.
Comments
Comment #2
Wim Leers+1! Dozens of times I've had to re-test patches for this. If this would be possible, this would really help :)
Comment #3
jonathan1055 CreditAttribution: jonathan1055 as a volunteer commentedYes I agree this would be great. There is a current problem on 7.x patch testing, which has been failing with 'CI error' since 15 Dec #2837015: CI problem with permission when testing 7.x patches. Having issues not revert to NW would be good in this situation.
Comment #4
Wim LeersThis is at least major.
This exact same problem happened again overnight. The entire 8.x RTBC queue was marked NW.
These are just the 5 RTBC Drupal core 8.x issues that I've been involved in:
Comment #5
MixologicWork is underway to mostly eliminate CI error from happening at all and be more explicit about what those error conditions are, so many of these wont happen again, but they will still be test fails (patch doesnt apply, @group missing, etc)
The most recent bout of CI errors was due to the new deployment changing all of the signaling to reflect a testing application (0 = pass, 1 = failed tests, 2 = exceptions during testing and > 2 = abnormal error state).
The problem was that Jenkins saw anything > 0 as a FAILURE, and drupal.org is currently interpreting Failure to mean CI error, when it was really just some failed tests, or failed patches.
So, anyhow, we're working on it, but ideally CI errors become less common, and we'll have to address what *types* of test states should/should not mark an issue as needs work.
Comment #6
Wim LeersHappening again now.
Comment #7
drummComment #9
drummComment #10
Wim LeersOMG OMG OMG OMG!
Thank you!
This will save us so much pain!
Comment #11
drummThis has been deployed.
Comment #12
Wim Leers<3 <3 <3