Test bot apparently strips tags from fail/error message

Newer and easier-to-repro version here: #2869573-9: Remove usages of deprecated DrupalKernel::prepareLegacyRequest()

I thought maybe this was within the realm of d.o, but it's right here: http://cgit.drupalcode.org/drupalci_testbot/tree/src/DrupalCI/Build/Arti...

So when the testbot finds a result in the database put there by run-tests.sh, it strips HTML from the message. This is fine for security reasons as mentioned, but a pain in debugging time.

Of course, it's doing this in order to put the message into JUnit, which does not assume that the message is HTML or not, so this leads to a problem: Simpletest output assumes HTML, and PHPUnit output does not.

But let's assume we're going to favor PHPUnit. In that case, we'd just pass along the message without any encoding or decoding. Then it would be D.O's responsibility to encode any HTML characters for display.

We can fix this if/when PIFT switches to sanitizing the results.

I think this may be stripped before it gets to Drupal.

Wim showed an example in Slack:

For example: https://www.drupal.org/pift-ci-job/2293147 shows:

--- Expected
+++ Actual
@@ @@
-' '
+''

whereas locally I see

--- Expected
+++ Actual
@@ @@
-'<br> <p>'
+''

For this test, we parse https://dispatcher.drupalci.org/job/drupal_patches/109440/testReport/api.... In that, I see

--- Expected\n+++ Actual\n@@ @@\n-' '\n+''

So either Jenkins’s parsing is dropping it, or the thing Jenkins parses doesn’t have it.

It's the testbot doing it. https://git.drupalcode.org/project/drupalci_testbot/blob/HEAD/src/Drupal...

      // Set up our return array.
      $mapped_result[$test_group][$test_class][$test_method][] = array(
        'status' => $result['status'],
        'type' => $result['message_group'],
        'message' => strip_tags(htmlspecialchars_decode($result['message'], ENT_QUOTES)),
        'line' => $result['line'],
        'file' => $result['file'],
      );

So everything downstream from the testbot is getting the sanitized text.

We'd prefer to have the testbot *not* strip the HTML but instead encode it, or perhaps not even really process it other than to turn it into legal JSON or XML or whatever.

That way any consumer would get an accurate message for users.

Then downstream consumers of the JUnit would be able to process it as needed to present to users in a safe way. Probably by encoding to entities.

Presumably the testbot is stripping out dangerous stuff because there was a need to do that. So we should ensure that downstream consumers (Jenkins, d.o) know that test results could have unencoded HTML, and are acting appropriately.

Then we can come back here to the testbot and change it to not strip the HTML.