Problem/Motivation

This issue is a child of #2680713: Handle missing @groups

In that issue, we discovered that missing @group errors are swallowed by the testbot because they are echo'd to stdout, but the testbot is piping stdout to a file.

In order to fix this, we have to change run-tests.sh to output errors to stderr instead.

This issue is scoped for changing the output of run-tests.sh --list in this way.

Proposed resolution

Have run-tests.sh output exception errors using error_log().

Remaining tasks

Change other output of run-tests.sh in a similar way, since it seems like a good idea anyway: #2808541: Make run-tests.sh better behaved with error_log()

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#3 2807271_3_test.patch814 bytesMile23
#2 2807271_2.patch369 bytesMile23
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Mile23 created an issue. See original summary.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Active » Needs review
FileSize
2807271_2.patch369 bytes

Le patch.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
FileSize
2807271_3_test.patch814 bytes

Here's a patch which should fail due to missing @group, alongside the changes to run-tests.sh.

We should see a missing @group message in the jenkins console log.

EDIT: And we do. See https://dispatcher.drupalci.org/job/default/222797/console

[..]
15:41:38 php /var/www/html/core/scripts/run-tests.sh  --list --php /opt/phpenv/shims/php > /var/www/html/artifacts/testgroups.txt
15:41:38 Command created as exec id 3a3a4824
15:41:39 exception 'Drupal\simpletest\Exception\MissingGroupException' with message 'Missing @group annotation in Drupal\Tests\Core\Access\AccessManagerTest' in /var/www/html/core/modules/simpletest/src/TestDiscovery.php:351
15:41:39 Stack trace:
15:41:39 #0 /var/www/html/core/modules/simpletest/src/TestDiscovery.php(177): Drupal\simpletest\TestDiscovery::getTestInfo('Drupal\\Tests\\Co...', '/**\n * @coversD...')
15:41:39 #1 /var/www/html/core/modules/simpletest/simpletest.module(588): Drupal\simpletest\TestDiscovery->getTestClasses(NULL, Array)
15:41:39 #2 /var/www/html/core/scripts/run-tests.sh(76): simpletest_test_get_all(NULL)
15:41:39 #3 {main}
15:41:39 HTTP/1.1 200 OK
15:41:39 Content-Type: application/vnd.docker.raw-stream
15:41:39 
15:41:39 
15:41:39 Error
15:41:39 Received a non-zero return code from the last command executed on the container.  (Return status: 2)
15:41:39 Execution Error
15:41:39 Error encountered while executing job build step execute:command
15:41:39 Checking console output
15:41:39 Recording test results
15:41:39 ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
15:41:40 Finished: FAILURE

Status: Needs review » Needs work

The last submitted patch, 3: 2807271_3_test.patch, failed testing.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Needs work » Needs review
Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Status: Needs review » Reviewed & tested by the community

This will save a lot of support burden for the DrupalCI maintainers.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
CreditAttribution: alexpott at Chapter Three commented
Status: Reviewed & tested by the community » Fixed
Issue tags: +rc eligible

Since this is a change to run-tests.sh and directly improves peoples ability to contribute with 0 risks to a working site I think this is rc eligible.

Committed and pushed c802506 to 8.3.x and 1878499 to 8.2.x. Thanks!

  • alexpott committed c802506 on 8.3.x 
    Issue #2807271 by Mile23: run-tests.sh needs to output an exception to...

  • alexpott committed 1878499 on 8.2.x 
    Issue #2807271 by Mile23: run-tests.sh needs to output an exception to...
Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Issue summary: View changes
Issue tags: +Dublin2016, +run-tests.sh
Related issues: +#2626986: [meta] Improvements to run-tests.sh

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.