Problem/Motivation

\Drupal\Tests\Listeners\DeprecationListenerTrait::getSkippedDeprecations() allows us to skip deprecations. In an ideal world we wouldn't have this. But we have to content with twig 2 -> 3 deprecations and sf 4 -> 5 and 5 -> 6 etc. And sometimes we won't be able to manage the change without a major release so the previous release might have to skip such deprecations.

Proposed resolution

Allow skipping deprecations in unit tests - like we can for other types of testing.

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

N/a

Comments

alexpott created an issue. See original summary.

alexpott’s picture

alexpott’s picture

StatusFileSize
new1.55 KB
new5.58 KB

Test only patch is the interdiff.

The last submitted patch, 3: 3031577-3.test-only.patch, failed testing. View results

gábor hojtsy’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests +Twig 3, +Drupal 9
larowlan’s picture

Status: Reviewed & tested by the community » Fixed

Fixed on commit

diff --git a/core/tests/Drupal/Tests/Listeners/AfterSymfonyListener.php b/core/tests/Drupal/Tests/Listeners/AfterSymfonyListener.php
index c374335ae9..121d0d2db2 100644
--- a/core/tests/Drupal/Tests/Listeners/AfterSymfonyListener.php
+++ b/core/tests/Drupal/Tests/Listeners/AfterSymfonyListener.php
@@ -8,7 +8,7 @@

 if (class_exists('PHPUnit_Runner_Version') && version_compare(\PHPUnit_Runner_Version::id(), '6.0.0', '<')) {
   class_alias('Drupal\Tests\Listeners\Legacy\AfterSymfonyListener', 'Drupal\Tests\Listeners\AfterSymfonyListener');
-  // Using an early return instead of a else does not work when using the
+  // Using an early return instead of an else does not work when using the
   // PHPUnit phar due to some weird PHP behavior (the class gets defined without
   // executing the code before it and so the definition is not properly
   // conditional).
diff --git a/core/tests/Drupal/Tests/Listeners/DeprecationListenerTrait.php b/core/tests/Drupal/Tests/Listeners/DeprecationListenerTrait.php
index 37e08914be..1103450a9b 100644
--- a/core/tests/Drupal/Tests/Listeners/DeprecationListenerTrait.php
+++ b/core/tests/Drupal/Tests/Listeners/DeprecationListenerTrait.php
@@ -165,8 +165,8 @@ protected function registerErrorHandler($test) {

     // Register another listener so that we can remove the error handler before
     // Symfony's DeprecationErrorHandler checks that it is the currently
-    // registered handler. Note this done like this to ensure the error hander
-    // is removed after SymfonyTestsListenerTrait::endTest() is called.
+    // registered handler. Note this is done like this to ensure the error
+    // handler is removed after SymfonyTestsListenerTrait::endTest() is called.
     // SymfonyTestsListenerTrait has its own error handler that needs to be
     // removed before this one.
     $test_result_object = $test->getTestResultObject();

Committed c52175d and pushed to 8.7.x. Thanks!

  • larowlan committed c52175d on 8.7.x
    Issue #3031577 by alexpott: \Drupal\Tests\Listeners\...
alexpott’s picture

Status: Fixed » Needs work
+++ b/core/tests/Drupal/Tests/Listeners/DeprecationListenerTrait.php
@@ -13,9 +13,21 @@
+    // @todo comment

Whoops

Also when running a test like core/modules/aggregator/tests/src/Functional/AddFeedTest.php I'm seeing THE ERROR HANDLER HAS CHANGED! outputted at the end of the test. That's not right. Going to revert to address this.

  • alexpott committed 812c0bb on 8.7.x
    Revert "Issue #3031577 by alexpott: \Drupal\Tests\Listeners\...
gábor hojtsy’s picture

wim leers’s picture

gábor hojtsy’s picture

Discussed this today, may help people working on this issue:

gaborhojtsy [2:00 PM]
I admit I did not understand what was the problem with the error handler swapping, is that too excessively happening where it should not? — I could work on the comment missing of course but that does not in itself help >D

alexpott [2:01 PM]
The symfony deprecation handler has a check in its shutdown function to ensure its expected error handler is still the error handler. I thought I’d managed to get this right by restoring the correct error handler before it runs but I missed something.

gaborhojtsy [2:02 PM]
ah, so we may be missing errors reported then(?)

alexpott [2:04 PM]
This is made way more complex by the fact we have quite a few layers to contend with. Isolated tests for example run in their own process. And all kernel / functional tests are isolated.
It looks like it’s the isolated tests where there is an issue.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new1.95 KB
new5.56 KB

Okay I've tried to reproduce what I saw after this was committed. I've run core/modules/aggregator/tests/src/Functional/AddFeedTest.php all ways up via run-tests.sh and phpunit and it's not happening for me. I have no idea how I saw what I was seeing. But DrupalCI tests did not fail and THE ERROR HANDLER HAS CHANGED! does not appear in DrupalCI output anywhere so I think we should assume that this was down to an environment error on my behalf. Sorry for the jittery revert - I think it might have been because all the mess with Extension objects in 8.6.x had me on edge for making a wide ranging mistake.

Here's a patch with the erroneous @todo removed. What a comment there would say is covered in way more depth by this comment that the patch already contains:

    // Register another listener so that we can remove the error handler before
    // Symfony's DeprecationErrorHandler checks that it is the currently
    // registered handler. Note this is done like this to ensure the error
    // handler is removed after SymfonyTestsListenerTrait::endTest() is called.
    // SymfonyTestsListenerTrait has its own error handler that needs to be
    // removed before this one.

The patch also includes @larowlan's on commit fixes from #6.

gábor hojtsy’s picture

Status: Needs review » Reviewed & tested by the community

Superb, thanks! Let's get this in for the 8.7 beta then :)

mikelutz’s picture

Status: Reviewed & tested by the community » Needs review

@alexpott Is it possible you were working on this patch in conjunction with symfony 4 testing, and were seeing the errors when you updated your dependencies to symfony 4.2? I am still seeing "THE ERROR HANDLER HAS CHANGED" locally with this patch, but only when using symfony 4 libraries. I'm seeing it in RecursiveContextualValidatorTest at least, at the moment.

lendude’s picture

StatusFileSize
new40.33 KB

I can reproduce #16 using the latest patch here and the latest patch on #2976394: Allow Symfony 4.4 to be installed in Drupal 8

lendude’s picture

StatusFileSize
new5.96 KB

This happens on all tests with multiple test methods. I think because the restore_error_handler() is done after each method, but we were only setting the new error handler on the start of the test run. Making sure that set_error_handler($deprecation_handler); is done for every test method seems to clear this up.

lendude’s picture

StatusFileSize
new1.66 KB

missing interdiff

catch’s picture

Nice find with #18. Marking RTBC. Kicked off a couple of extra test runs.

catch’s picture

Status: Needs review » Reviewed & tested by the community
alexpott’s picture

StatusFileSize
new891 bytes
new5.97 KB

Fixing PHP7.3-dev test. We don't need to register an error handler if \Drupal\Tests\Listeners\DeprecationListenerTrait::deprecationStartTest() is called for a test suite. This should only be done when we're handling a real test.

catch’s picture

Happy to commit this once #22 comes back green.

mikelutz’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs reroll
alexpott’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new6.21 KB

Rerolled.

mikelutz’s picture

Issue tags: -Needs reroll
catch’s picture

Version: 8.8.x-dev » 8.7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed 51cc668 and pushed to 8.8.x. Thanks!

Moving to 'to be ported' against 8.7.x for cherry-pick after the beta freeze.

  • catch committed 51cc668 on 8.8.x
    Issue #3031577 by alexpott, Lendude, larowlan: \Drupal\Tests\Listeners\...
catch’s picture

Status: Patch (to be ported) » Fixed

Cherry-picked to 8.7.x

  • catch committed 2d7507a on 8.7.x
    Issue #3031577 by alexpott, Lendude, larowlan: \Drupal\Tests\Listeners\...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.