Hello,

When using multiple sorts, the order of the sorts is not respected.

When using the short syntax, sorts are added in the reverse order:

sort=-created,uid.name

Will give the uid.name added first.

When using the normal syntax, sorts are added in the alphabetical order of their ID.

Before trying to make a patch: Is this behavior expected?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Grimreaper created an issue. See original summary.

e0ipso’s picture

This is not the expected behavior. Thanks for reporting @Grimreaper!

e0ipso’s picture

Before trying to make a patch: Is this behavior expected?

I'll try to find someone to work on this during sprints in the Decoupled Drupal Dev Days. If you already have some code in progress please assign the issue to yourself.

Grimreaper’s picture

Hello @e0ipso,

No code in progress yet, I have been on other subjects.

Thanks

dawehner’s picture

Category: Support request » Bug report
Status: Active » Needs review
FileSize
2.65 KB

After writing the test I believe the problem is that \Drupal\jsonapi\Query\QueryBuilder::buildTree doens't keep the insert order around, by popping instead of shifting.

Status: Needs review » Needs work

The last submitted patch, 5: 2898155-5.patch, failed testing. View results

e0ipso’s picture

@dawehner that patch only contains the failing test (which is very helpful). Just checking that you didn't forget to submit any code (since the patch filename is not suffixed with _tests_only).

dawehner’s picture

This is right so far ... I have't tried to fix it yet.

Grimreaper’s picture

Status: Needs work » Fixed

Hello,

After seeing #2921892: fix(Sort): Expanded form sort parameters should respect order, I have retested with the last dev version of JSONAPI, and it is fixed now.

Grimreaper’s picture

Status: Fixed » Needs review

I requeued patch from dawehner, maybe it can be merged to have more tests.

Status: Needs review » Needs work

The last submitted patch, 5: 2898155-5.patch, failed testing. View results

e0ipso’s picture

Issue tags: +Needs reroll

@Grimreaper that patch failed to apply. Can you re-roll?

Grimreaper’s picture

Assigned: Unassigned » Grimreaper

@e0ipso: ok, assigning to myself. I will make the re-roll tomorrow.

Grimreaper’s picture

Status: Needs work » Needs review
FileSize
3.25 KB

Here is the rebased patch. I also fixed some coding standards at the same time.

But the tests fails:

There was 1 failure:

1) Drupal\Tests\jsonapi\Functional\JsonApiFunctionalTest::testRead
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
Array (
- 0 => 55
- 1 => 60
- 2 => 59
- 3 => 58
- 4 => 57
- 5 => 56
+ 0 => 1
+ 1 => 2
+ 2 => 3
+ 3 => 4
+ 4 => 5
+ 5 => 6
)

/project/www/core/tests/Drupal/Tests/BrowserTestBase.php:1240
/project/www/modules/contrib/jsonapi/tests/src/Functional/JsonApiFunctionalTest.php:294

I will see why. I have uploaded this failing patch to be able to make an interdiff.

Status: Needs review » Needs work

The last submitted patch, 14: jsonapi-test_multiple_sorts-2898155-14.patch, failed testing. View results

Grimreaper’s picture

Assigned: Grimreaper » Unassigned
Status: Needs work » Needs review
FileSize
3.27 KB
899 bytes

Here is the test fixed.

But to fix it, there is no sort in DESC. So I don't know if the node samples should be modified to keep having "field_sort1,-field_sort2".

Thanks for the review.

Wim Leers’s picture

Title: Multiple sort: order of sorts not respected » Test coverage: multiple sorts
Component: Documentation » Code
Issue tags: -Needs reroll +API-First Initiative
Related issues: +#2921892: fix(Sort): Expanded form sort parameters should respect order, +#2874601: refactor(QueryBuilder): Improve testability/maintainability
FileSize
4.87 KB
4.42 KB

Thanks, @Grimreaper! 👏

Since this was fixed elsewhere, this is now a test-only issue. Updating title accordingly.

  1. +++ b/tests/src/Functional/JsonApiFunctionalTestBase.php
    @@ -129,6 +129,34 @@ abstract class JsonApiFunctionalTestBase extends BrowserTestBase {
    +    $field_storage_config = FieldStorageConfig::create([
    ...
    +    $field_storage_config->save();
    ...
    +    $field_config = FieldConfig::create([
    ...
    +    $field_config->save();
    

    Nit: We never use these temporary variables. So might as well do FieldConfig::create([…])->save();

  2. +++ b/tests/src/Functional/JsonApiFunctionalTestBase.php
    @@ -243,6 +271,13 @@ abstract class JsonApiFunctionalTestBase extends BrowserTestBase {
    +      // Create values for the sort field.
    +      // Let field_sort1 just increase every 5 items, but field_sort2 decrease
    +      // constantly.
    

    Comment formatting and spelling needs to be improved.

    Love the code though!

  3. +++ b/tests/src/Functional/JsonApiFunctionalTest.php
    @@ -281,6 +281,17 @@ class JsonApiFunctionalTest extends JsonApiFunctionalTestBase {
    +        'sort' => 'field_sort1,field_sort2',
    

    We should also test all other combinations. This is testing both in ASC order.

Addressed all 3 points.

Wim Leers’s picture

FileSize
724 bytes
4.42 KB

Darn, a change I made for debugging end up in #17.

The last submitted patch, 17: 2898155-17.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Wim Leers’s picture

In writing that additional test coverage, I think I ended up finding a bug (and that's why I had that debug leftover accidentally in #17, fixed in #18):

  1. +++ b/tests/src/Functional/JsonApiFunctionalTest.php
    @@ -281,6 +281,54 @@ class JsonApiFunctionalTest extends JsonApiFunctionalTestBase {
    +        'page[limit]' => 6,
    +        'sort' => 'field_sort1,field_sort2',
    ...
    +    $this->assertCount(6, $output_nids);
    

    👌 Limit = 6, ASC+ASC, output = 6.

  2. +++ b/tests/src/Functional/JsonApiFunctionalTest.php
    @@ -281,6 +281,54 @@ class JsonApiFunctionalTest extends JsonApiFunctionalTestBase {
    +        'page[limit]' => 6,
    +        'sort' => 'field_sort1,-field_sort2',
    ...
    +    $this->assertCount(6, $output_nids);
    

    👌 Limit = 6, ASC+DESC, output = 6.

  3. +++ b/tests/src/Functional/JsonApiFunctionalTest.php
    @@ -281,6 +281,54 @@ class JsonApiFunctionalTest extends JsonApiFunctionalTestBase {
    +        'page[limit]' => 6,
    +        'sort' => '-field_sort1,field_sort2',
    ...
    +    $this->assertCount(5, $output_nids);
    

    🐛 Limit = 6, DESC+ASC, output = 5!?

  4. +++ b/tests/src/Functional/JsonApiFunctionalTest.php
    @@ -281,6 +281,54 @@ class JsonApiFunctionalTest extends JsonApiFunctionalTestBase {
    +        'page[limit]' => 6,
    +        'sort' => '-field_sort1,-field_sort2',
    ...
    +    $this->assertCount(5, $output_nids);
    

    🐛 Limit = 6, DESC+DESC, output = 5?!

Am I right in thinking that is wrong, or am I missing something?

Grimreaper’s picture

Assigned: Unassigned » Grimreaper

Hello,

@Wim Leers, thanks for your feedbacks and for updating the patch.

I am triggering the tests locally and yes, the output count not matching the limit is strange. I am investigating.

Grimreaper’s picture

I am verifying but I think I found why, at the beginning of the test method:

  // Unpublish the last entity, so we can check access.
  $this->nodes[60]->setUnpublished()->save();

But if the user can't access the last entity it should display another one published, no?

Grimreaper’s picture

Assigned: Grimreaper » Unassigned

OK.

I confirm that it is this line that provoked the fact that the node 61 is excluded from the result but included in the query limit.

Otherwise the expect nid order would be:

    // 24. Test sort criteria on multiple fields: first DESC, second ASC.
...
    $this->assertEquals([61, 60, 59, 58, 57, 56], $output_nids);
...
    // 25. Test sort criteria on multiple fields: both DESC.
...
    $this->assertEquals([61, 56, 57, 58, 59, 60], $output_nids);

I quickly reproduced the behavior:

With 3 articles: article 1, article 2, article 3

The article 3 is unpublished.

On jsonapi/node/article as anonymous, I see two articles and a meta error message:

    "meta": {
        "errors": [
            {
                "title": "Forbidden",
                "status": 403,
                "detail": "The current user is not allowed to GET the selected resource.",
                "links": {
                    "info": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4"
                },
                "code": 0,
                "id": "/node--article/8ed78306-ff14-4ef2-9eb7-1091cf98bbf5",
                "source": {
                    "pointer": "/data"
                }
            }
        ]
    },

On jsonapi/node/article?page[limit]=1&sort=-title as anonymous, I only see with no data:

    "meta": {
        "errors": [
            {
                "title": "Forbidden",
                "status": 403,
                "detail": "The current user is not allowed to GET the selected resource.",
                "links": {
                    "info": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4"
                },
                "code": 0,
                "id": "/node--article/8ed78306-ff14-4ef2-9eb7-1091cf98bbf5",
                "source": {
                    "pointer": "/data"
                }
            }
        ]
    },

Also there is a need to add a stop at the end of the comment (for PHPCS):

      // Create values for the sort fields, to allow for testing complex
      // sorting:
      // - field_sort1 increments every 5 articles, starting at zero
      // - field_sort2 decreases every article, ending at zero.
e0ipso’s picture

@Grimreaper are you saying that no data is returned at all if there is one item with access denied in the result set, and there is a sort criteria? That's troubling.

Grimreaper’s picture

@e0ipso: sorry, the manual test case is ambiguous.

There is no data because I limit the result to 1 item and with the sort criteria it is the unpublished item that is the first one. But as the anonymous user can't access this unpublished node, there is no data.

If there is no result limit, the anonymous user sees 2 articles (the two published) with a meta errors for the unpublished one.

e0ipso’s picture

Status: Needs review » Needs work

Ah! That makes sense.

Can we make sure we are explicit about that and have:

$this->assertCount(5, $output_nids);
// Field node:XX is unpublished.
$this->assertCount(1, $output['meta']['errors]);
Wim Leers’s picture

#26++

Wim Leers’s picture

Wim Leers’s picture

FileSize
1.12 KB
4.53 KB

Implemented #26.

Grimreaper’s picture

Hello,

Thanks both for your replies and work.

But is it an expected behavior?

I mean, when having a view with a limit to only 5 results, if I have 100 published content and one unpublished. Even if the unpublished one should be in the results due to the sorts criteria, I want the visitors to see 5 results, so if other contents match the filters and the visitor has access to the other contents, its should be displayed.

Maybe I should open a new issue for that? Because this issue is about sorts, not filtering.

e0ipso’s picture

In that scenario you should filter your list by the published flag.

There is not much that can be done at the query level (that's what selects your results) that evaluates custom PHP logic in your results. So custom access checks are applied at a later stage. This is a limitation that will not be overcome in the foreseeable future.

  • e0ipso committed c8f9a18 on 8.x-1.x authored by Grimreaper
    Issue #2898155 by Wim Leers, Grimreaper, dawehner, e0ipso: Test coverage...
e0ipso’s picture

Status: Needs review » Fixed
Grimreaper’s picture

OK.

Thanks for the reply and the commit.

Wim Leers’s picture

Very glad to have this one fixed! It's exactly this sort of test coverage that helps build trust in this module's stability, and also helps document expected behavior.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.