#3039568-60: Add a read-only mode to JSON:API added an update path test: ReadOnlyModeUpdateTest
. It passes tests fine locally.
JSON:API's test suite passes just fine, both on #3039568: Add a read-only mode to JSON:API and on https://www.drupal.org/node/2723491/qa.
But when ported to core, it fails: #2843147-171: Add JSON:API to core as a stable module.
Queue surprise.
I re-tested the core patch. Consistently failing. I re-tested locally. Consistently passing. In different PHP installations.
Queue despair.
Locally, I was testing with the PHPUnit test runner. DrupalCI is testing with run-tests.sh
. It took me 45 minutes of debugging run-tests.sh
to figure out the correct incantation to run a single class. 😡 php core/scripts/run-tests.sh --url http://d8.test --module jsonapi --suppress-deprecations --non-html --concurrency "12"
has worked fine for months, but replacing --module jsonapi
with --group jsonapifoobar
doesn't work, nor does --file path/to/file
, nor does --class F/Q/C/N
. Turns out that for everything except --module
, it needs to be at the end of the arguments … (ノಥ益ಥ)ノ ┻━┻ 🤯
But even php core/scripts/run-tests.sh --url http://d8.test --suppress-deprecations --non-html --concurrency "12" --file core/modules/jsonapi/tests/src/Functional/Update/ReadOnlyModeUpdateTest.php
didn't work. I had to debug run-tests.sh
further, to figure out why I was getting ERROR: No valid tests were specified.
. At some point, it checks if is_subclass_of($namespace_class, TestCase::class)
. That turned out to be FALSE too. Which led to me inspecting the class hierarchy, comparing with other core test that now did work appropriately, and then I saw it.
👁
-namespace Drupal\jsonapi\Tests\Functional\Update;
+namespace Drupal\Tests\jsonapi\Functional\Update;
😭
And yes, inspecting the test runs for #3039568 and the project-level test runs, I can see that the new test simply isn't being run. This is very easy to miss though.
It's still a mystery to me why only the core test runner detects this problem. I wonder how many contrib modules have tests that do work fine locally, in their maintainers' IDE, yet don't actually run on DrupalCI at all…
Comment | File | Size | Author |
---|---|---|---|
#4 | 3040186-4.patch | 837 bytes | Wim Leers |
| |||
#4 | interdiff.txt | 634 bytes | Wim Leers |
#2 | 3040186-2.patch | 574 bytes | Wim Leers |
Comments
Comment #2
Wim LeersComment #4
Wim LeersFailed solely due to deprecation notices, all triggered by Drupal core.
We'll follow the example of core's update path tests: add
@group legacy
and@group Update
. 🤞Comment #5
Wim LeersVICTORY!
Comment #7
Wim Leers