TestBot throws uncaught PHP Warning on 11.x-dev only

Problem/Motivation

On any testrun on 11.x-dev there's a PHP warning visible in the console output:

PHP Warning: A non-numeric value encountered in /var/www/html/core/tests/Drupal/Tests/Core/Extension/InfoParserUnitTest.php on line 301

@andypost already spotted this in #3134417: Add \Drupal::getMajorVersion()

This warning doesn't cause the test to fail, nor is it reported anywhere.
That can't be right, right?

Steps to reproduce

- Go to https://www.drupal.org/node/3060/qa
- Click on any of the test runs in 11.x-dev
- Click on "View results on dispatcher"
- Click on "View as plain text"
- Do a search on "A non-numeric value encountered"

The PHP warning can be seen around this region:

[0m[32mDrupal\Tests\Core\Form\EventSubscriber\FormAjaxSubscriberTes   7 passes                                      
[0m[32mDrupal\Tests\Core\EventSubscriber\PsrResponseSubscriberTest    2 passes                                      
[0m[32mDrupal\Tests\Core\EventSubscriber\OptionsRequestSubscriberTe 152 passes                                      
[0m[32mDrupal\Tests\Core\Extension\DefaultConfigTest                  1 passes                                      
[0m[32mDrupal\Tests\Core\Extension\DependencyTest                     6 passes                                      
[0m[32mDrupal\Tests\Core\Extension\ExtensionVersionTest             102 passes                                      
[0m[32mDrupal\Tests\Core\Extension\ExtensionSerializationTest         2 passes                                      
[0m[32mDrupal\Tests\Core\Extension\ExtensionListTest                 19 passes                                      
[0mPHP Warning:  A non-numeric value encountered in /var/www/html/core/tests/Drupal/Tests/Core/Extension/InfoParserUnitTest.php on line 301
[32mDrupal\Tests\Core\Extension\LegacyExtensionTest                1 passes                                      
[0m[32mDrupal\Tests\Core\Extension\ModuleInfoTest                    78 passes                                      
[0m[32mDrupal\Tests\Core\Extension\InfoParserUnitTest                31 passes                                      
[0m[32mDrupal\Tests\Core\Extension\ExtensionDiscoveryTest             3 passes                                      
[0m[32mDrupal\Tests\Core\Extension\RequiredModuleUninstallValidator   3 passes                                      
[0m[32mDrupal\Tests\Core\EventSubscriber\ActiveLinkResponseFilterTe 2853 passes                                      
[0m[32mDrupal\Tests\Core\Extension\ModuleRequiredByThemesUninstallV   3 passes                                      
[0m[32mDrupal\Tests\Core\Field\PluginSettingsBaseTest                 1 passes                                      
[0m[32mDrupal\Tests\Core\Field\FieldDefinitionListenerTest            5 passes                                      
[0m[32mDrupal\Tests\Core\Field\FieldInputValueNormalizerTraitTest    14 passes                         

Proposed resolution

Make sure that when (at the very least this) PHP Warnings are thrown, they are logged more visibly
Currently they only whiz by when you look at the console log of a test run, not afterwards.

Decide if tests should fail when a PHP warning is logged.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet