#3039568-60: Add a read-only mode to JSON:API added an update path test: ReadOnlyModeUpdateTest. It passes tests fine locally.

JSON:API's test suite passes just fine, both on #3039568: Add a read-only mode to JSON:API and on https://www.drupal.org/node/2723491/qa.

But when ported to core, it fails: #2843147-171: Add JSON:API to core as a stable module.

Queue surprise.

I re-tested the core patch. Consistently failing. I re-tested locally. Consistently passing. In different PHP installations.

Queue despair.

Locally, I was testing with the PHPUnit test runner. DrupalCI is testing with run-tests.sh. It took me 45 minutes of debugging run-tests.sh to figure out the correct incantation to run a single class. 😡 php core/scripts/run-tests.sh --url http://d8.test --module jsonapi --suppress-deprecations --non-html --concurrency "12" has worked fine for months, but replacing --module jsonapi with --group jsonapifoobar doesn't work, nor does --file path/to/file, nor does --class F/Q/C/N. Turns out that for everything except --module, it needs to be at the end of the arguments … (ノಥ益ಥ)ノ ┻━┻ 🤯

But even php core/scripts/run-tests.sh --url http://d8.test --suppress-deprecations --non-html --concurrency "12" --file core/modules/jsonapi/tests/src/Functional/Update/ReadOnlyModeUpdateTest.php didn't work. I had to debug run-tests.sh further, to figure out why I was getting ERROR: No valid tests were specified.. At some point, it checks if is_subclass_of($namespace_class, TestCase::class). That turned out to be FALSE too. Which led to me inspecting the class hierarchy, comparing with other core test that now did work appropriately, and then I saw it.




👁

-namespace Drupal\jsonapi\Tests\Functional\Update;
+namespace Drupal\Tests\jsonapi\Functional\Update;

😭

And yes, inspecting the test runs for #3039568 and the project-level test runs, I can see that the new test simply isn't being run. This is very easy to miss though.

It's still a mystery to me why only the core test runner detects this problem. I wonder how many contrib modules have tests that do work fine locally, in their maintainers' IDE, yet don't actually run on DrupalCI at all…

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers created an issue. See original summary.

Wim Leers’s picture

Status: Active » Needs review
FileSize
574 bytes

Status: Needs review » Needs work

The last submitted patch, 2: 3040186-2.patch, failed testing. View results

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
634 bytes
837 bytes

Failed solely due to deprecation notices, all triggered by Drupal core.

We'll follow the example of core's update path tests: add @group legacy and @group Update. 🤞

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

VICTORY!

  • Wim Leers committed d5abca9 on 8.x-2.x
    Issue #3040186 by Wim Leers: Follow-up for #3039568:...
Wim Leers’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.