Results broken when run-tests exits with appropriate error code

Bumping the priority to (hopefully) get some attention.

We have made a branch to address this in drupalci here is the run
https://dispatcher.drupalci.org/job/staging_simpletest_job/323/console

Thanks a lot!
So, the script seems to run fine, but jenkins throws an error at the end:

01:32:22 Checking console output
01:32:22 Recording test results
01:32:24 Build step 'Publish JUnit test result report' changed build result to UNSTABLE
01:32:24 ERROR: Failed to set Jenkins Identity header on email.
01:32:24 java.lang.NullPointerException
01:32:24 	at org.jenkinsci.main.modules.instance_identity.InstanceIdentity.get(InstanceIdentity.java:126)
01:32:24 	at jenkins.plugins.mailer.tasks.MimeMessageBuilder.setJenkinsInstanceIdent(MimeMessageBuilder.java:188)
01:32:24 	at jenkins.plugins.mailer.tasks.MimeMessageBuilder.buildMimeMessage(MimeMessageBuilder.java:166)
01:32:24 	at hudson.tasks.MailSender.createEmptyMail(MailSender.java:391)
01:32:24 	at hudson.tasks.MailSender.createUnstableMail(MailSender.java:221)
01:32:24 	at hudson.tasks.MailSender.createMail(MailSender.java:183)
01:32:24 	at hudson.tasks.MailSender.run(MailSender.java:107)
01:32:24 	at hudson.tasks.Mailer.perform(Mailer.java:141)
01:32:24 	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:74)
01:32:24 	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
01:32:24 	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:776)
01:32:24 	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
01:32:24 	at hudson.model.Build$BuildExecution.post2(Build.java:183)
01:32:24 	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:670)
01:32:24 	at hudson.model.Run.execute(Run.java:1763)
01:32:24 	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
01:32:24 	at hudson.model.ResourceController.execute(ResourceController.java:98)
01:32:24 	at hudson.model.Executor.run(Executor.java:381)
01:32:24 Sending e-mails to: ryan@association.drupal.org
01:32:25 Finished: UNSTABLE

Can we do anything to fix this?

Also it differs from the error in the last test run from #2189345: run-tests.sh should exit with a failure code if any tests failed where it says:

20:34:27 Error
20:34:27 Received a non-zero return code from the last command executed on the container.  (Return status: 1)
20:34:27 Execution Error
20:34:27 Error encountered while executing job build step execute:command
20:34:27 Checking console output
20:34:27 Recording test results
20:34:27 ERROR: Publisher 'Publish JUnit test result report' failed: No test report files were found. Configuration error?
20:34:27 Finished: FAILURE

Any ideas?

@sanduhrs: that NPE error is a problem with jenkins emailing plugin.. its nothing we can really fix short of patching the java plugin: https://issues.jenkins-ci.org/browse/JENKINS-26740

And, yes, it differs, as the output you are seeing in the other issue is what we're fixing. Drupalci currently aborts processing when it sees an exit status of anything other than 0.

The code in #5 is not in production drupalci yet, its only available for ad hoc test runs that we run internally like #6, to establish that our changes are going to work going forward. Since they appear to be functioning correctly, marking this needs review for eventual merge into the dev branch.

Thanks for clarification!

#5 looks good, please merge.

The patch look good to me.

This is deployed to the testbots.

Great, thanks allot!