Problem/Motivation

We have a Drupal Commerce checkout using an off-site iframe for payment. It pings a callback controller and with some other logic we load a route using a ?refid= query parameter. Despite this value changing, or being present and then not, the page is HIT after first view by dynamic page cache

Data from Chrome dev tools network tab:

First request:

Request URL:https://example.com/checkout/zuora/2804/payment?refid=
Request Method:GET
Status Code:302 

cache-control:must-revalidate, no-cache, private
content-language:en
content-type:text/html; charset=UTF-8
date:Wed, 10 May 2017 21:32:35 GMT
expires:Sun, 19 Nov 1978 05:00:00 GMT
location:/checkout/2804/payment
server:nginx
status:302
x-drupal-dynamic-cache:MISS

Second request

Request URL:https://example.com/checkout/zuora/2804/payment?refid=REDACTED_VALUE
Request Method:GET
Status Code:302 

cache-control:must-revalidate, no-cache, private
content-language:en
content-type:text/html; charset=UTF-8
date:Wed, 10 May 2017 21:32:35 GMT
expires:Sun, 19 Nov 1978 05:00:00 GMT
location:/checkout/2804/payment
server:nginx
status:302
x-drupal-dynamic-cache:HIT

Since the refid was present, it should have had a different location header.

For now, I have disabled cache on this route. But I had to move this callback out of our checkout plugin and into a custom route with caches disabled.

Proposed resolution

No idea. \Drupal\Core\Cache\Context\RouteCacheContext checks the route parameters. Which is what the $dynamicPageCacheRedirectRenderArray uses. It also checks request format.

Maybe also add url.query_args as render context?

Remaining tasks

User interface changes

API changes

Data model changes

Comments

mglaman created an issue. See original summary.

mglaman’s picture

Issue summary: View changes
mglaman’s picture

I had to add no_cache: TRUE to the route, also, even though it returns a redirect response like so:

    $url = Url::fromRoute('commerce_checkout.form', ['commerce_order' => $order->id(), 'step' => $step_id])->toString(TRUE);
    $response = new TrustedRedirectResponse($url->getGeneratedUrl());
    // Added this when dependencies didn't do anything.
    $response->setMaxAge(0);
    $response->addCacheableDependency($url);
    $response->addCacheableDependency($order);
    return $response;
mglaman’s picture

Status: Active » Needs review
StatusFileSize
new1.3 KB

While reviewing the test, and wanting to write one, I discovered this:

      // Finally, let's also verify that the 'dynamic_page_cache_test.html'
      // route continued to see cache hits if we specify a query argument,
      // because it *should* ignore it and continue to provide Dynamic Page
      // Cache hits.
      $url = Url::fromUri('route:dynamic_page_cache_test.html', ['query' => ['animal' => 'piglet']]);
      $this->drupalGet($url);
      $this->assertEqual('HIT', $this->drupalGetHeader(DynamicPageCacheSubscriber::HEADER), 'Render array returned, rendered as HTML response: Dynamic Page Cache is active, Dynamic Page Cache HIT.');

So this is intended? What if the query adjusts a page's dynamic content. Such as gtm parameters?

mglaman’s picture

Hoping Fabianx or Wim Leers can provide insight why queries (different ones) wouldn't bust cache in regards to personalization, etc.

Status: Needs review » Needs work

The last submitted patch, 4: dynamic_cache_does_not-2877498-4.patch, failed testing.

wim leers’s picture

Category: Bug report » Support request
Status: Needs work » Postponed (maintainer needs more info)
Issue tags: -Needs subsystem maintainer review

A response MUST vary based on route (because a different controller is called for each route) and request format (again, different controller, or at least different post-processing for the same controller's return value).

Any variations beyond that are handled automatically by Dynamic Page Cache. On one condition: your controller specifies the correct cacheability metadata.

Everything in this issue suggests that Drupal Commerce is missing some cacheability metadata: it should specify

$build['#cache']['contexts'] = 'url.query_args:refid';

in the code building a render array which varies depending on the specified ?refid in the URL.

mglaman’s picture

But we're not returning a renderable array. See #3. We're returning a HTTP redirect response, because we're handling an off-site iframe response. Adding cache metadata to that response seems to have no effect. Or do I need to merge in url.query_args:refid (even with max age at zero?)

I guess I should write a test for core which properly mimics what we're doing.

wim leers’s picture

Issue tags: +D8 cacheability

Sorry, I missed #3.

    $url = Url::fromRoute('commerce_checkout.form', ['commerce_order' => $order->id(), 'step' => $step_id])->toString(TRUE);
    $response = new TrustedRedirectResponse($url->getGeneratedUrl());
    // Added this when dependencies didn't do anything.
    $response->setMaxAge(0);
    $response->addCacheableDependency($url);
    $response->addCacheableDependency($order);
    return $response;

So this adds the cache tag for the Order entity. It also sets the cache tags + cache contexts that are bubbled during URL generation. Likely you're getting the url.site cache context. But how would $url know about this refid URL query argument from the incoming request, which is causing this redirect?

From what you described, you need to add

$response->getCacheableMetadata()->addCacheContexts(['url.query_args:refid'])

Ideally, I would be able to see the entire controller logic, then I can answer this properly.

mglaman’s picture

Status: Postponed (maintainer needs more info) » Closed (works as designed)

This works as designed,then, as outlined by the test around

$url = Url::fromUri('route:dynamic_page_cache_test.html.with_cache_contexts', ['query' => ['animal' => $animal]]);
wim leers’s picture

So that means you were able to get it working as expected, by adding the missing cacheability metadata? :)

wim leers’s picture

Status: Closed (works as designed) » Needs review
StatusFileSize
new3.75 KB

Hm, but wait:

    $response->setMaxAge(0);

That should have ensured the response wasn't cached by Dynamic Page Cache. We even have test coverage for that in \Drupal\Tests\dynamic_page_cache\Functional\DynamicPageCacheIntegrationTest:

    // Max-age = 0 responses are ignored by Dynamic Page Cache.
    $this->drupalGet('dynamic-page-cache-test/html/uncacheable/max-age');
    $this->assertEqual('UNCACHEABLE', $this->drupalGetHeader(DynamicPageCacheSubscriber::HEADER), 'Render array returned, rendered as HTML response, but uncacheable: Dynamic Page Cache is running, but not caching.');

If that's not working, then that would be a bug. Test to prove that this indeed should work (existing test coverage should already be sufficient, but since this issue is suggesting it doesn't actually work, adding more explicit test coverage).

wim leers’s picture

mglaman’s picture

Sorry, yes! This has fixed our problems.

wim leers’s picture

Correcting the cacheability metadata has fixed your problems?

mglaman’s picture

It worked on my non-form GET requests. I have yet to test it with forms, and ensuring forms with the attached metadata are not cached. I can take some time to re-roll #12 to add this test. Just have been busy lately.

wim leers’s picture

Status: Needs review » Postponed (maintainer needs more info)

Well, existing test coverage should be sufficient, but I'd love to hear it if that weren't the case.

Marking as postponed on the feedback #16 indicates will be provided in the future :)

drupalninja99’s picture

@Wim I am trying this within a block that I would think would bubble up to the page? I am using both the getCacheMaxAge() method (setting it to zero) and getCacheContexts() using 'url.query_args' but the page cache is still ignoring the query string. I am using page_manager which should also respect cache contexts but maybe the bug is in page manager (in which case I can log an issue there)?

wim leers’s picture

Note page_cache !== dynamic_page_cache. I think it's best if you open a new issue and the actual code (sanitized). And page_manager has been known to have many cacheability bugs, so yes, that's quite likely.

jribeiro’s picture

Just a feedback, I have a pretty similar case described in this issue and adding $response->getCacheableMetadata()->addCacheContexts(['url.query_args:query']) solved the issue for me, dynamic cache is caching both urls, with and without the query string.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Pinged @mglaman yesterday to get feedback per #17.

mglaman’s picture

Sorry for the delay. The 'url.query_args context is working properly for any and all use cases.

For example I added

    $build['#cache']['contexts'][] = 'url.query_args:refid';
    $build['#cache']['contexts'][] = 'url.query_args:zuoraEc';
    $build['#cache']['contexts'][] = 'url.query_args:zuoraEm';

And it bubbles up breaking form cache.

wim leers’s picture

Status: Postponed (maintainer needs more info) » Closed (works as designed)

Alright, thanks for confirming! 👍

floriane’s picture

I had this problem with my controller's module, which took the cache value instead of the url one (i'm using Drupal 10)

I solved the problem with #cache and max-age to 0:

        $address = $request->query->get('adresse');

        return [
            '#address' => $address,
            '#cache' => [
                'max-age' => 0,
            ],
        ];