I have a custom entity type, "task", which has an entity reference field to a content type (nodes) "project".

I'm using a progressively-decoupled setup. Listing tasks using views and using jsonAPI have both always worked fine.

I recently implemented node_grants, and simply declaring the hook (even without returning anything) breaks my ability to query tasks using jsonAPI.

The error message is the dreaded:
LogicException: The controller result claims to be providing relevant cache metadata, but leaked metadata was detected. Please ensure you are not rendering content too early. Returned object class: Drupal\jsonapi\ResourceResponse. in Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext() (line 154 of /var/www/dev-1/XX/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php).

The custom entity type is fairly vanilla - it was originally set up using Drupal Console. And this breaks even without fully implementing node_grants; declaring it is enough to break it.

I've spent a long time digging into this and the only way I've been able to get around the issue is deleting the (premature?) rendering of nodes:

diff --git a/core/modules/node/node.module b/core/modules/node/node.module
index 9fdabf32b4..dcda495519 100644
--- a/core/modules/node/node.module
+++ b/core/modules/node/node.module
function node_query_node_access_alter(AlterableInterface $query) {
   $request = \Drupal::requestStack()->getCurrentRequest();
   $renderer = \Drupal::service('renderer');
   if ($request->isMethodCacheable() && $renderer->hasRenderContext()) {
     $build = ['#cache' => ['contexts' => ['user.node_grants:' . $op]]];
-    $renderer->render($build);
   }
 }

I know that's a security risk because it prevents cache tags from bubbling up / risk of users seeing each other's data / bypassing access.

What I would *like* to do, but can't find a way to is something like this:

diff --git a/core/modules/node/node.module b/core/modules/node/node.module
index 9fdabf32b4..dcda495519 100644
--- a/core/modules/node/node.module
+++ b/core/modules/node/node.module
@@ -1119,8 +1119,9 @@ function node_query_node_access_alter(AlterableInterface $query) {
   $request = \Drupal::requestStack()->getCurrentRequest();
   $renderer = \Drupal::service('renderer');
   if ($request->isMethodCacheable() && $renderer->hasRenderContext()) {
+    $render_context = $renderer->getCurrentRenderContext();
     $build = ['#cache' => ['contexts' => ['user.node_grants:' . $op]]];
-    $renderer->render($build);
+    $render_context->update($build);
   }
 }

That isn't possible because getCurrentRenderContext is a private method.

Am I totally off-base here? Any other solution? Like I said, everything works perfectly fine until I create a *literally empty* (or even properly populated) node_grants function. If I rename that function to something else (so that I am not using node_grants anymore), all of the tasks load fine.

-----
EDIT/UPDATE: See the attached patch "3032041-node_grant_premature_render-2.patch" for an updated potential solution which the above led me to.

I now believe the issue is with Renderer::hasRenderContext() returning true even if the context returns a count of 0.

Comments

ashrafabed created an issue. See original summary.

ashrafabed’s picture

Status: Active » Needs review
StatusFileSize
new623 bytes

After lots and lots of digging through core code, I found a solution that works for me and has no downsides AFAIK. Please take a look.

ashrafabed’s picture

Issue summary: View changes
ashrafabed’s picture

Issue summary: View changes

Status: Needs review » Needs work

The last submitted patch, 3: 3032041-node_grant_premature_render-2.patch, failed testing. View results

timmillwood’s picture

Issue tags: +Needs tests

Looks like it works for you, but not for a number of core tests.

I feel I don't know the node_grants, rendering, or JSON API well enough, but would making getCurrentRenderContext a public method help?

We will also need tests to recreate your use case, but without JSON API module in core.

hchonov’s picture

The custom entity type is fairly vanilla - it was originally set up using Drupal Console. And this breaks even without fully implementing node_grants; declaring it is enough to break it.

That happens because if there are no grants defined, then node_query_node_access_alter() will exit early and not reach the render part at the end of the function:

if (!count(\Drupal::moduleHandler()->getImplementations('node_grants'))) {
    return;
  }

The rendering part was added in #2557815: Automatically bubble the "user.node_grants:$op" cache context in node_query_node_access_alter(). It is interesting to note that it has a condition:

if ($request->isMethodCacheable() && $renderer->hasRenderContext()) {

However $renderer->hasRenderContext() will always return TRUE if called inside a http request, because the render context is started in \Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber::wrapControllerExecutionInRenderContext() for every http request.

If we execute a request to simply retrieve some raw data, then why do we start a render context? Is there a way to prevent this? E.g. what if we flag the request in a way that it does not expect any rendering to occur and if that happens, then throw an exception? For this we could wrap the controller in another method inside \Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber::onController() which then will put the renderer service in a state preventing rendering.

ashrafabed’s picture

"would making getCurrentRenderContext a public method help?"
@7timmillwood that's what I tried first. I was able to get the context that way, then to call the ->update() method and pass the render array to the update method. I thought that would fix it, but the update method crashed when it called $this->pop(); inside of the RenderContext class. I saw that it was crashing because $this->count() had 0 items. And that led me to noticing hasRenderContext() was returning true regardless of whether there were any items returned from ->count().

"That happens because if there are no grants defined, then node_query_node_access_alter() will exit early and not reach the render part at the end of the function:"
@hchonov I only commented the grants out for testing. It is also happening when there are grants defined.

"However $renderer->hasRenderContext() will always return TRUE if called inside a http request"
I think this is the real problem. My patch makes it so that, in my manual tests, it only returns TRUE when actually rendering a node, but it returns false when rendering my custom entity type which has a reference field to a node. I'm curious as to why it's causing the other tests to fail; admittedly I don't know enough about what is being counted.

I feel this could possibly be reproduced in core if you have a taxonomy type with a reference field to nodes, *and* you have a custom node_grants implementation (or even an empty hook_node_grants, apparently). I'll have to test that when I have some time - already sunk 5 hours into this issue so far and it's a busy week / the patch is getting the job done for me so far.

"If we execute a request to simply retrieve some raw data, then why do we start a render context? Is there a way to prevent this? "
I looked all over for a way to "bubble up" the cache context without explicitly rendering and couldn't find anything.

wim leers’s picture

+++ b/core/lib/Drupal/Core/Render/Renderer.php
@@ -567,7 +567,11 @@ protected function doRender(&$elements, $is_root_call = FALSE) {
-    return (bool) $this->getCurrentRenderContext();
+    if ((bool) $this->getCurrentRenderContext()) {
+      $context = $this->getCurrentRenderContext();
+      return (bool) ($context->count() > 0);
+    }
+    return FALSE;

This is wrong; this bypasses the protection of \Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber. This is the long-term solution: #3028976: Enable an entity query's return value to carry cacheability.

breaks my ability to query tasks using jsonAPI.

Please provide steps to reproduce. Which version of JSON:API are you using? This used to be a bug in JSON:API a long time ago, we've fixed it since then, in #2984964: JSON API + hook_node_grants() implementations: accessing /jsonapi/node/article as non-admin user results in a cacheability metadata leak.

ashrafabed’s picture

This is happening in JSON:API 2.1.0.

I can reproduce it on my site by visiting: /jsonapi/ENTITY_TYPE/ENTITY_BUNDLE .

I'm going to try to create steps for reproducing it with one of core's entity types now. Not sure it's possible, but we'll see.

ashrafabed’s picture

@Wim I wasn't successful reproducing it in core in the hour I was able to devote to this today. I know that I'll need to provide steps to reproduce for someone to be able to help further.

I did have one question / wanted to see what I'm missing here:
"This is wrong; this bypasses the protection of \Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber."

I am curious how that patch bypasses anything - this patch is functionally the same:

+++ b/core/lib/Drupal/Core/Render/Renderer.php
@@ -567,7 +567,11 @@ protected function doRender(&$elements, $is_root_call = FALSE) {
-    return (bool) $this->getCurrentRenderContext();
+    return ((bool) $this->getCurrentRenderContext() && (bool) $this->getCurrentRenderContext()->count());

With my (admittedly limited) context, it seems like it isn't bypassing anything. I thought writing the patch in a more clear way might be helpful so I could get confirmation that this approach is a bad one.

More to the point: Are there risks associated with using that ^ code snippet until I can either get to the bottom of the issue or at least properly reproduce it?

wim leers’s picture

this patch is functionally the same

I don't see why that's functionally the same. If it's the same, then why change it? This went from a single condition to two conditions. The second condition is effectively saying "Oh only do this if something already was added to the render context, don't do this if it would be the first thing to do so". That's why it's bypassing the protection.

Are there risks associated with using that ^ code snippet until I can either get to the bottom of the issue or at least properly reproduce it?

There is a risk that your cached response is not going to be varying correctly, and hence serve data cached for user X to user Y.

wim leers’s picture

Status: Needs work » Postponed (maintainer needs more info)
ashrafabed’s picture

I meant the patch I attached to the ticket was functionally the same as the way I re-wrote it in my last comment. Not that it was the same as core's previous functionality.

But this answered my question: "Oh only do this if something already was added to the render context, don't do this if it would be the first thing to do so".

Planning to circle back on this in a few weeks due to time constraints, will use the patch until then.

wim leers’s picture

👌

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

amber himes matz’s picture

Issue tags: +Bug Smash Initiative, +jsonapi

Thank you for reporting this issue and working with other community members to try and understand and resolve the problem.

As part of the Bug Smash Initiative, we are triaging issues that are marked "Postponed (maintainer needs more info)". This issue was marked "Postponed (maintainer needs more info)" back in February 2019.

There has been no activity here for 3 years and 3 months.

Since we need more information (steps to reproduce) to move forward with this issue, I am keeping the status at Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after 3 months.

Thanks!

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mxr576’s picture

Status: Postponed (maintainer needs more info) » Needs work
Related issues: +#3052553: Entity query alter with cacheable metadata leaks and triggers LogicException

#3052553: Entity query alter with cacheable metadata leaks and triggers LogicException is kinda related just early rendering in the db query layer happens elsewhere.

The problem with the fix that was added in #2984964: JSON API + hook_node_grants() implementations: accessing /jsonapi/node/article as non-admin user results in a cacheability metadata leak that it only covers node-[node type] collection routes and _nothing more_. That was a fix rather a band aid until #3028976: Enable an entity query's return value to carry cacheability.

I have also bumped into this issue on user/user endpoint when I implemented a custom access checker that only granted access to a user's username if that user is author of a node that the current acting user has view access. (Related to #3241232: [policy] Treat username enumerations as security bugs that require Security Advisories) My code fired an entity query like below. There was no issue with this query until I enabled a module that provides a hook_node_grants() implementation, like node_access_test module, because node_query_node_access_alter() only then starts performing early rendering.

    $query = $this->nodeStorage->getQuery();
    $query->accessCheck(TRUE);
    $query->addMetaData('account', $acting_user);
    $query->addMetaData('op', 'view');
    $query->condition('uid', $author->id());
    return $query->execute();

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

acbramley’s picture

Status: Needs work » Postponed (maintainer needs more info)
Issue tags: +Needs issue summary update, +Needs title update

We need a proper update to the IS and title here. Also it sounds like this might be fixed by #3028976: Enable an entity query's return value to carry cacheability? Is this a duplicate?

ashrafabed’s picture

As the original reporter of this issue, I only faced this issue on one project and I haven't been able to reproduce it since.

As far as I am concerned, I am OK with closing the issue unless someone else is still facing it and can provide more information.

smustgrave’s picture

Status: Postponed (maintainer needs more info) » Closed (outdated)

Since there hasn't been a follow up I'm going to close out but am leaving all the credit assigned.