When a patch fails to apply, jenkins aborts the job, and no junit.xml is output. This causes drupalci to mark them as a 'Testbot Error' (white) vs as a red 'Failure' - a failed patch application should show up as a patch failure on d.o.

CommentFileSizeAuthor
#9 2534144-failed-patches.patch2.35 KBMixologic
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Related issues: -#2534132: Disable Legacy Testbots and use drupalCI as our testing infrastructure
Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Parent issue: » #2534132: Disable Legacy Testbots and use drupalCI as our testing infrastructure
Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Title: Failed patches should show as fails on drupal.org » Patches that fail to apply should show as fails on drupal.org
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
CreditAttribution: hestenet at Drupal Association commented

Ideally this should be represented on the dispatcher in junit xml output that can be parsed like any other result data.

This could be accomplished by the test runner itself validating whether the test completed and aborting if any particular step (such as applying the patch) fails.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
CreditAttribution: hestenet at Drupal Association commented
Related issues: +#2558929: Testrunner code cleanup and refactoring

Per jthorson, logic changes in #2558929: Testrunner code cleanup and refactoring should help to capture failed build steps as failures.

jthorson’s picture

jthorson CreditAttribution: jthorson for Drupal Association commented

Those changes lay the groundwork, and provide a mechanism to stop execution upon failure ... the actual failure logic still needs to be built into the individual build step plugins. I can't remember if the key method is on $job or the class returned by $job->getJobResults().

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Status: Active » Needs review

This has been brutally hacked in for the time being - i.e. its a straight xml string that gets spit out into the xml directory if a patch fails to apply. But its better than clicking through.

For some reason the commits arent showing up properly here, but heres a test run: https://dispatcher.drupalci.org/job/DCI_XML/132/testReport/

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented

Okay, here's an example of how it will look on drupal.org for the time being.

https://staging.devdrupal.org/pift-ci-job/33336

I think we can commit this now, and then drumm can later use the Testsuite Error message to bubble up the Patch Fail message.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
FileSize
2534144-failed-patches.patch2.35 KB

and heres a patch for review.

basic’s picture

basic CreditAttribution: basic at Drupal Association commented
Status: Needs review » Reviewed & tested by the community

Looks good.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Status: Reviewed & tested by the community » Fixed

Aight this hot mess hack is now in. Good.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.