Problem/Motivation

PHPUnit does not include deprecations nor warnings in the JUnit output.

See Include deprecation/warning failures in JUnit logs

run-tests.sh in the parent issue #2905007: Allow run-tests.sh to report skipped/incomplete PHPUnit tests has been changed to be more adherent to JUnit logs in its report output - and reports as passed runs that throw such deprecations or warnings.

Proposed resolution

Look at exit codes of the PHPUnit CLI processes on top of the JUnit logs to report as failed tests that throw deprecations or warnings.

Screenshots - note that the screenshot refer to a test whose context was changed to lead to a test run failure, but the test itself is not in need of fixes:

Before

After

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#5 after.png914.74 KBmondrake
#5 before.png909.18 KBmondrake

Issue fork drupal-3535637

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

mondrake created an issue. See original summary.

mondrake opened merge request !12703

mondrake’s picture

mondrake
Location 🇮🇹
commented
Title: Improve run-tests.sh output when deprecations an warnings are triggered » Improve run-tests.sh output when deprecations and warnings are triggered
mondrake’s picture

mondrake
Location 🇮🇹
commented
Status: Active » Needs review
mondrake’s picture

mondrake
Location 🇮🇹
commented
Issue summary: View changes
StatusFileSize
new before.png909.18 KB
new after.png914.74 KB
godotislate’s picture

godotislate
he/him
commented

Haven't had a chance yet to test locally what the junit looks like, but had a question about the need for and placement of the update hook.

mondrake’s picture

mondrake
Location 🇮🇹
commented

Replied inline. Yes, it's odd. Hopefully one day we will get rid of the test results database and of the --sqlite argument in the run-tests.sh script.

godotislate’s picture

godotislate
he/him
commented
Status: Needs review » Reviewed & tested by the community

OK, tested locally, LGTM.

11.x

SIMPLETEST_DB="sqlite://localhost/sites/default/files/db.sqlite" php -d memory_limit=1G core/scripts/run-tests.sh --verbose --sqlite test.sqlite --color --types PHPUnit-Unit Lock

Drupal test run

Drupal Version:   11.3-dev
PHP Version:      8.3.21
PHP Binary:       /usr/bin/php
PHPUnit Version:  11.5.22
-------------------------------

Tests to be run:
  - Drupal\Tests\Core\Lock\LockBackendAbstractTest

Test run started:
  Tuesday, July 15, 2025 - 09:30

Test summary
------------

   2.180s Drupal\Tests\Core\Lock\LockBackendAbstractTest                            3 passed, 1 log(s)

Test run duration: 1 sec

Detailed test results
---------------------


---- Drupal\Tests\Core\Lock\LockBackendAbstractTest ----


Status      Duration Info
--------------------------------------------------------------------------------------------------------
Pass          0.033s testWaitFalse
Pass          1.007s testWaitTrue
Pass          0.002s testGetLockId
Log           1.139s *** Process execution output ***
    PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

    Runtime:       PHP 8.3.21
    Configuration: /var/www/html/core/phpunit.xml

    ...                                                                 3 / 3 (100%)

    Time: 00:01.044, Memory: 6.00 MB

    Lock Backend Abstract (Drupal\Tests\Core\Lock\LockBackendAbstract)
     ✔ Wait false
     ✔ Wait true
     ✔ Get lock id

Merge request !12703

SIMPLETEST_DB="sqlite://localhost/sites/default/files/db.sqlite" php -d memory_limit=1G core/scripts/run-tests.sh --verbose --sqlite test.sqlite --color --types PHPUnit-Unit Lock

Drupal test run

Drupal Version:   11.3-dev
PHP Version:      8.3.21
PHP Binary:       /usr/bin/php
PHPUnit Version:  11.5.22
-------------------------------

Tests to be run:
  - Drupal\Tests\Core\Lock\LockBackendAbstractTest

Test run started:
  Tuesday, July 15, 2025 - 09:29

Test summary
------------

   2.195s Drupal\Tests\Core\Lock\LockBackendAbstractTest                            3 passed, exit code 1

Test run duration: 1 sec

Detailed test results
---------------------


---- Drupal\Tests\Core\Lock\LockBackendAbstractTest ----


Status      Duration Info
--------------------------------------------------------------------------------------------------------
Pass          0.033s testWaitFalse
Pass          1.008s testWaitTrue
Pass          0.002s testGetLockId
Failure       1.152s *** Process execution output ***
    PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

    Runtime:       PHP 8.3.21
    Configuration: /var/www/html/core/phpunit.xml

    ...                                                                 3 / 3 (100%)

    Time: 00:01.046, Memory: 6.00 MB

    Lock Backend Abstract (Drupal\Tests\Core\Lock\LockBackendAbstract)
     ✔ Wait false
     ✔ Wait true
     ✔ Get lock id

  • catch committed 4fd90637 on 11.x 
    Issue #3535637 by mondrake, godotislate: Improve run-tests.sh output...

catch closed merge request !12703

catch’s picture

catch
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Fixed

Thanks for linking #2905007: Allow run-tests.sh to report skipped/incomplete PHPUnit tests because I'd forgotten about that and would have had to go through all the same thought process all over again. Apart from not adding an update at all, and potentially breaking people's local installs, we don't have much choice here.

This will help prevent a lot of confusion for a lot of people.

Committed/pushed to 11.x, thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.