Patch plugin doesn't signal an application error

The patchfactory addition and all the other bits is great, the problem, however, is that the patch returning 0 right now is deliberately broken:

-        //throw $e;
-        return 0;
+        throw $e;

This is because right now if jenkins sees any return code other than 0 from drupal ci, it marks the build as FAILED - which in turn gets converted into a "CI Error" by d.o. when it retrieves the results via the API. So to fix *that*, we need to make sure that our buildstate.json actually has something worth telling drupal.org about in it. i.e. if a patch fails, we can/should be putting the exception message into buildstate.json. Once that file has something d.o. can rely on, it wont have to report "CI Error" when it sees a FAILED. Ic an simply report whatever it sees in that file (which is either structured, or *just* has the "short statement that fills the red pill on d.o. testing results"

Since we launched drupalci improved, we have the following error messages in buildstate.json:

4 "Ancillary require failure.  Error Code: 2"
2670 "Build Successful"
  94 "Composer require failure.  Error Code: 2"
  13 "Composer require failure.  Error Code: 255"
   1 "PHPLint Failed: \n\nEXECUTING: cd \/var\/www\/html && xargs -P 31 -a \/var\/lib\/drupalci\/artifacts\/lintable_files.txt -I {} php -l '{}'\n\nPHP Parse error:  syntax error, unexpected '$this' (T_VARIABLE) in \/var\/www\/html\/core\/modules\/views\/tests\/src\/FunctionalJavascript\/ExposedFilterAJAXTest.php on line 118\nxargs: php: exited with status 255; aborting\n"
   2 "PHPLint Failed: \n\nEXECUTING: cd \/var\/www\/html && xargs -P 31 -a \/var\/lib\/drupalci\/artifacts\/lintable_files.txt -I {} php -l '{}'\n\nPHP Parse error:  syntax error, unexpected ')' in \/var\/www\/html\/core\/modules\/block_content\/src\/Tests\/BlockContentListViewsTest.php on line 106\nxargs: php: exited with status 255; aborting\n"
  10 "PHPLint Failed: \n\nEXECUTING: cd \/var\/www\/html && xargs -P 31 -a \/var\/lib\/drupalci\/artifacts\/lintable_files.txt -I {} php -l '{}'\n\nPHP Parse error:  syntax error, unexpected ')' in \/var\/www\/html\/modules\/contrib\/panels\/src\/Plugin\/DisplayBuilder\/StandardDisplayBuilder.php on line 69\nxargs: php: exited with status 255; aborting\n"
   1 "PHPLint Failed: \n\nEXECUTING: cd \/var\/www\/html && xargs -P 31 -a \/var\/lib\/drupalci\/artifacts\/lintable_files.txt -I {} php -l '{}'\n\nPHP Parse error:  syntax error, unexpected '[' in \/var\/www\/html\/sites\/all\/modules\/examples\/dbtng_example\/dbtng_example.module on line 634\nxargs: php: exited with status 255; aborting\n"
   1 "PHPLint Failed: \n\nEXECUTING: cd \/var\/www\/html && xargs -P 31 -a \/var\/lib\/drupalci\/artifacts\/lintable_files.txt -I {} php -l '{}'\n\nPHP Parse error:  syntax error, unexpected '}' in \/var\/www\/html\/modules\/webform\/src\/WebformElementBase.php on line 1426\nxargs: php: exited with status 255; aborting\n"

So I see a couple things wrong there. We're saying too much in the Lint Exceptions, the Error codes from composer dont tell us anything, and the ancillary require failures are like , uh, what?.

Also, patch fails arent showing up there at all.. Even from before the time when it was throwing exceptions, so I think the exception message got swallowed on the first catch, and the second throw didnt say much. Its almost as if catching and rethrowing drained it of its exception message somehow.

So, step one is Fix what ends up in buildstate.json (and yeah, name to buildoutcome.json or buildresult.json. the state isnt going to change once its over.).
Step two is massage d.o's project issue file test to care about buildresult.json when it sees FAILURE at jenkinsland.
Step three, then would be to make patch properly throw an exception, and remove any notion that it needs to spew out some xml in order to trick jenkins into making an UNSTABLE build with one failed test.

Finally we can then add some other exception situations on our side like look for missing @group, look for 'No tests found', and set the buildstate accordingly. (essentially anything that yields a CI error now).

We pretty much want to get to a world where CI error only ever happens when something totally craps the bed, like we push a broken drupacl branch to production or something.

We have liftoff. Drupal.org is now properly introspecting buildoutcome.json.

We can now do proper signaling throughout the application, and that the patch plugin no longer needs to fake an xml test result.

howdy. This is now deployed, and patches fail the build properly. Yay.