From Slack:

gabesullice [10:02 AM]
@e0ipso @wimleers, for the core patch, we needed to test JSON:API against Postgres. It looks like since we don't have any default sorting and some of our tests don't specify a sort order, we're getting failures because Postgres is not matching the sort order we expect (which is what Mysql would return by default). I also have this problem locally w/ `UserTest` because I have MySQL 8 running.

I think we have three choices:

1. Add a default sort order
2. Add logic to our tests to account for non-deterministic sort order
3. Add sorts to all our test queries

1. could be bad because it could change implicit expectations that people might have based on how JSON:API works today
2. Could be bad because it adds even _more_ complexity to our tests
3. Could be bad because we'll never test the "not sorted" case

My preference is to go w/ #1, because it is the simplest solution from a maintenance perspective. It could be viewed as a feature because responses will be uniformly sorted across different databases. I think the risk of actually breaking assumptions is pretty low. _Maybe_ some hardcoded tests that people have will break, but the solution is simple: fix the test's order/add a sort to the request

Comments

gabesullice created an issue. See original summary.

wim leers’s picture

I also have this problem locally w/ `UserTest` because I have MySQL 8 running.

That means we'd run into this problem in the future even without Postgres. Core is keeping us honest :)

It could be viewed as a feature because responses will be uniformly sorted across different databases.

+1

I think the risk of actually breaking assumptions is pretty low. _Maybe_ some hardcoded tests that people have will break, but the solution is simple: fix the test's order/add a sort to the request

+1

gabesullice’s picture

StatusFileSize
new238 bytes
new475 bytes

Update: syntax errors mask the expected failures. To save you clicks... MySQL 8 failed UserTest and Postgres failed many.

The first patch changes nothing, it just gives us something to prove the "bug" with MySQL 8 and Postgres.

The second commit is the simplest change. It might break some tests.

e0ipso’s picture

#1 incurs into a performance penalty, so -1 for me. I like #3 over others.

Could be bad because we'll never test the "not sorted" case

Tests assert that sorting works. Unsorted defaults to DB implementation (not on us to test). We only need to test that an unsorted collection works. That could be a dedicated test.

gabesullice’s picture

StatusFileSize
new3.6 KB

I really wonder what the impact of ordering by an integer primary key really has, but I also admit that #1 being the easiest to implement was a major factor in my decision (which is a polite way of saying, "my laziness affects my decision-making").

Here's an attempt at option 3. I think this will solve for MySQL 8, but it might not address all the PostgreSQL errors yet.

Update: that's frustrating, I'm running MySQL 8 locally and do not get the same errors with UserTest.

wim leers’s picture

#1 incurs into a performance penalty, so -1 for me.

I hadn't thought about that. This is an excellent point. 👏 🙏

@e0ipso convinced me we want #2 or #3.

gabesullice’s picture

StatusFileSize
new3.43 KB
new4.52 KB

I think this should fix MySQL 8. On the bright side, #5 did better on Postgres than I expected :)

gabesullice’s picture

StatusFileSize
new3.84 KB
new8.36 KB
gabesullice’s picture

StatusFileSize
new7.82 KB
new15.74 KB

I was a little baffled by this one on PostgreSQL:

1) Drupal\Tests\jsonapi\Functional\JsonApiFunctionalTest::testRead
Failed asserting that 49 matches expected 50.

/var/www/html/core/tests/Drupal/Tests/BrowserTestBase.php:699
/var/www/html/modules/contrib/jsonapi/tests/src/Functional/JsonApiFunctionalTest.php:34

Since it's asserting a collection count, not an ID or something. I realized that it's because an unpublished node was appearing on a different page than expected because of a different ordering. RuntimeAccessChecks--

gabesullice’s picture

Status: Active » Needs review
wim leers’s picture

Assigned: Unassigned » e0ipso
Status: Needs review » Reviewed & tested by the community

So the patch is green for #3. @e0ipso expressed his preference for this approach. Great!

I have just one concern: we're now coupling many of our tests to sorting. On the one hand this is completely reasonable, on the other hand it means there is a very slight risk in that we're no longer testing the absence of sorts. However …

+++ b/tests/src/Functional/ResourceTestBase.php
@@ -1038,6 +1038,20 @@ abstract class ResourceTestBase extends BrowserTestBase {
+    // This asserts that collections will work without a sort, added by default
+    // below, without actually asserting the content of the response.
...
+    $response = $this->request('HEAD', $collection_url, $request_options);
...
+    // Different databases have different sort orders, so a sort is required so
+    // test expectations do not need to vary per database.
+    $default_sort = ['sort' => 'drupal_internal__' . $this->entity->getEntityType()->getKey('id')];
+    $collection_url->setOption('query', $default_sort);

Gabe already thought about that :)

Given that, I'm RTBC'ing this, but not yet committing: I'd like @e0ipso to explicitly +1 this too.

wim leers’s picture

Title: Add a default sort order when one is not provided. » Add sorts to our collection tests that are inspecting values in the collection, to not be dependent on per-DB default order
Category: Feature request » Task
Priority: Normal » Critical
e0ipso’s picture

+1

wim leers’s picture

Status: Reviewed & tested by the community » Fixed

🎉

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.