1. Fresh install drupal8.5.x,jsonapi,devel,basic_auth
  2. Create two authenticated user: jim and odin
  3. Give "View own unpublished content" and "Article: Create new content"permission to authenticated user
  4. Use jim to create one unpublished article
  5. Clear cache
  6. use odin GET /jsonapi/node/article through postman, we get empty result.
    {
        "data": [],
        "jsonapi": {
            "version": "1.0",
            "meta": {
                "links": {
                    "self": "http://jsonapi.org/format/1.0/"
                }
            }
        },
        "links": {
            "self": "http://cache.n8/jsonapi/node/article"
        },
        ...
    }
      
  7. Then use jim to GET /jsonapi/node/article through postman, the data is still empty
    {
        "data": [],
        "jsonapi": {
            "version": "1.0",
            "meta": {
                "links": {
                    "self": "http://jsonapi.org/format/1.0/"
                }
            }
        },
        "links": {
            "self": "http://cache.n8/jsonapi/node/article"
        },
        ...
    }
      
  8. Clear cache
  9. Use jim to GET /jsonapi/node/article through postman we can get right result.
    {
        "data": [
            {
                "type": "node--article",
                "id": "32023099-4805-4be1-921a-9e03dd622561",
                "attributes": {
                    "nid": 1,
                    "uuid": "32023099-4805-4be1-921a-9e03dd622561",
                    "vid": 2,
                    "langcode": "en",
                    "revision_timestamp": "2018-06-30T15:30:17+00:00",
            ...
    }
    
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

caseylau created an issue. See original summary.

lawxen’s picture

Issue summary: View changes
lawxen’s picture

Title: cache problem with pager, If someone get empty data in the first page, then other user get empty result too. » cache problem with different user if someone get empty result.
Issue summary: View changes
Wim Leers’s picture

Unpublished to verify this is not a security vulnerability. It sounds like it is one.

Wim Leers’s picture

Title: cache problem with different user if someone get empty result. » NodeAccessControlHandler::checkAccess() does not add a necessary cache context
Project: JSON:API » Drupal core
Version: 8.x-1.x-dev » 8.6.x-dev
Component: Code » node system
Priority: Critical » Major
Issue tags: +D8 cacheability, +Entity Access, +API-First Initiative, +Contributed project soft blocker

Bug confirmed. The root cause is that the response contains a user.permissions cache context only, not a user cache context. This is why the response does not vary correctly between users odin and jim. Fortunately, it's not a security vulnerability AFAICT. So, republishing.


But why doesn't it? JSON API does correctly call entity access. But if jim created the unpublished article, and odin is the first to access it (to prime the cache), then an unfortunate combination of circumstances occurs in \Drupal\node\NodeAccessControlHandler::checkAccess():

    // Check if authors can view their own unpublished nodes.
    if ($operation === 'view' && !$status && $account->hasPermission('view own unpublished content') && $account->isAuthenticated() && $account->id() == $uid) {
      return AccessResult::allowed()->cachePerPermissions()->cachePerUser()->addCacheableDependency($node);
    }

    // Evaluate node grants.
    return $this->grantStorage->access($node, $operation, $account);

When odin first accesses it, the if-test evaluates to FALSE, and we evaluate node grants instead to determine access. Access is then denied, which is good, but it doesn't vary by user. It only varies by user.permissions. This leads to a perfectly cacheable response, so Dynamic Page Cache will cache it and … also serve it to any other user with the same set of permissions, which includes jim!

However, when jim (the owner) first access it, the if-test evaluates to TRUE. Access is granted, which is good, and the access result varies by user, as it should! In this case however, Dynamic Page Cache won't cache the response, because it deems responses varying by user not worthy of caching, since there's potentially millions of variations to cache (when there are millions of users).

Combined, this means that the owner's response will never be cached, but any non-owner's request will be. And after a non-owner has requested this, and the response is cached, then the same cached response will be sent to the owner. Users who aren't supposed to be able to access this data are never able to, but users who are supposed to have access to the data lose access as soon as any user who isn't has requested it!


The solution is trivial:

-    return $this->grantStorage->access($node, $operation, $account);
+    return $this->grantStorage->access($node, $operation, $account)->cachePerUser();
Wim Leers’s picture

Status: Active » Needs review
Issue tags: +Needs tests
FileSize
678 bytes
lawxen’s picture

@Wim Leers
The actual situation I met is group module's group_content, I just use core to reproduce the bug, so the patch won't solve our problem, I'll try to reproduced it on other entity type.

lawxen’s picture

I find this issue also happens on group_content, eck, commerce_order.

because group_content is complicated, so let's focus on node, eck, commerce_order first.
Same reproduced method: Enable the "view own ... " permission

Status: Needs review » Needs work

The last submitted patch, 6: 2982770-6.patch, failed testing. View results

lawxen’s picture

@Wim Leers
This issue is much more serious when it meet jsonapi's pager.

If user A request an endpoint and return empty in the first page. It will make user B's request get empty too. I have written an reproduce steps previously:

  1. Fresh install drupal8.5.x,jsonapi,devel,basic_auth
  2. Create two authenticated user: jim(uid:2) and odin(uid:3)
  3. Give "View own unpublished content" permission to authenticated user
  4. Request /devel/php and execute below code to generate test data(52 unpublished article whose author is jim and 1 unpublished article whose author is odin):
        for ($i = 1; $i <= 52; $i++){
          $article = Drupal\node\Entity\Node::create([
            'type' => 'article',
            'title' => 'article'.$i,
            'uid' => 2
          ]);
          $article->setUnpublished();
          $article->save();
        }
    
        $article = Drupal\node\Entity\Node::create([
          'type' => 'article',
          'title' => 'article'.'53',
          'uid' => 3
        ]);
        $article->setUnpublished();
        $article->save();
       
  5. Clear cache
  6. use odin GET /jsonapi/node/article through postman, we get empty array, this is right result, because odin can't access the front 50 article
    {
        "data": [],
        "jsonapi": {
            "version": "1.0",
            "meta": {
                "links": {
                    "self": "http://jsonapi.org/format/1.0/"
                }
            }
        },
        "links": {
            "self": "http://cache.n8/jsonapi/node/article",
            "next": "http://cache.n8/jsonapi/node/article?page%5Boffset%5D=50&page%5Blimit%5D=50"
        }
        ...
    }
      

    Use odin GET /jsonapi/node/article?page%5Boffset%5D=50&page%5Blimit%5D=50
    We can get one article

  7. Then use jim to GET /jsonapi/node/article through postman, the data is still empty array
    {
        "data": [],
        "jsonapi": {
            "version": "1.0",
            "meta": {
                "links": {
                    "self": "http://jsonapi.org/format/1.0/"
                }
            }
        },
        "links": {
            "self": "http://cache.n8/jsonapi/node/article",
            "next": "http://cache.n8/jsonapi/node/article?page%5Boffset%5D=50&page%5Blimit%5D=50"
        }
        ...
    }
      
  8. Clear cache
  9. Use jim to GET /jsonapi/node/article through postman we can get 50 articles.
lawxen’s picture

#5 Very detailed analysis. I solve our problem of group_content by the same way of #6

diff --git a/src/Entity/Access/GroupContentAccessControlHandler.php b/src/Entity/Access/GroupContentAccessControlHandler.php
index fd69e16..0315aa5 100644
--- a/src/Entity/Access/GroupContentAccessControlHandler.php
+++ b/src/Entity/Access/GroupContentAccessControlHandler.php
@@ -19,7 +19,7 @@ class GroupContentAccessControlHandler extends EntityAccessControlHandler {
    */
   protected function checkAccess(EntityInterface $entity, $operation, AccountInterface $account) {
     /** @var \Drupal\group\Entity\GroupContentInterface $entity */
-    return $entity->getContentPlugin()->checkAccess($entity, $operation, $account);
+    return $entity->getContentPlugin()->checkAccess($entity, $operation, $account)->cachePerUser();
   }
 
   /**

Maybe many contribute module should do the similar change.

@Wim Leers Besides, thanks for the so many guidance for me in drupal.org. I learned a lot from you.

Wim Leers’s picture

Everything in #10 sounds like just another consequence of the root cause I explained in #5. I completely agree the consequences can be big!

#11: adding ->cachePerUser() is not the solution that is always correct. It's important to look at the access control logic. It makes sense in the case of NodeAccessControlHandler::checkAccess() because it's first checking something that is true per user. In case of the Group module, there are severe known issues with cacheability, see #2952714: Group module's GroupAccessResult::allowedIfHasGroupPermission(s)() does not include cacheability.

@Wim Leers Besides, thanks for the so many guidance for me in drupal.org. I learned a lot from you.

I'm glad that you're finding it helpful! ❤️👍 I also want to thank you for creating such excellent bug reports, and helping to dig deeper to find the root cause, and then fix it properly! 👏 5 JSON API issues already in the past 12 months!

nassaz’s picture

First you have to give permission "Administer content" to Jim to create unpulished content.

After analysis, I can confirm that there is a problem of cache dependencies, and indeed it is necessary to add a context user but it's not the good place (cf. next patch)

nassaz’s picture

Status: Needs work » Needs review
FileSize
761 bytes

Check this patch.

Status: Needs review » Needs work

The last submitted patch, 14: 2982770-14.patch, failed testing. View results

nassaz’s picture

Issue tags: +DevDaysLisbon, +DevDaysLisboa
Wim Leers’s picture

@ntijani Thanks for working on this! It's good to have my conclusion verified. The patch in #14 is identical to the one I posted in #6 though. It'd be great if you could work on making tests pass! :)

nassaz’s picture

@Wim Leers thank you for your message.

You made the modification in the class "NodeAccessControlHandler" which is particular, I think it must be put in the method access() of the class "NodeGrantDatabaseStorage", i will try to make tests pass.

:)

chipway’s picture

Issue tags: -DevDaysLisboa

Removed DevDaysLisboa to find it in the same place.

nassaz’s picture

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Sara101’s picture

This issue seems old and abandoned but is similar to a problem I'm facing.
So admin users have permissions to view unpublished content.
If the admin user unpublished content and an unauthenticated user was first to visit the jsonapi node endpoint, then the unauthenticated user would not load content as expected.
But…
If the Admin user is the first to visit the jsonapi node endpoint after unpublishing the node, then unauthenticated users visiting same endpoint can also see the unpublished data until I manually clear cache(not expected)

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.