Problem/Motivation

https://qa.drupal.org/pifr/test/1076308

According the PHPDocs in the CacheContextsManager the calling method is REQUIRED to send an array of strings. Currently it is not doing so, and the assert statement reveals this. Whether or not this is an underlying cause of a greater issue is unknown.

Steps to reproduce

NA

Proposed resolution

- Assert that cache context keys and tokens are strings in CacheContextManager::convertTokensToKeys and ContextCacheKeys::__construct()
- In QueryArgsCacheContext::getContext(), when the value is an array, return a query string with the query argument and all values separated by "&", by using the existing http_build_query().

Remaining tasks

Review MR
Commit

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

N/A

Issue fork drupal-2509436

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

Aki Tendo’s picture

Status: Active » Needs review
StatusFileSize
new21.25 KB
new1.27 KB

Uploaded is a patch containing only the assert statement that is being failed, and the assert patch (so that the test can run)

Status: Needs review » Needs work

The last submitted patch, 1: 2509436-1.diff, failed testing.

dawehner’s picture

The only thing I can imagine that its NULL sometimes ...

      $this->resultsKey = $this->view->storage->id() . ':' . $this->displayHandler->display['id'] . ':results:' . hash('sha256', serialize($key_data));

at least this piece of code will always return in a string

olli’s picture

Component: views.module » cache system
Status: Needs work » Needs review
Issue tags: -bug, -views +VDC, +D8 cacheability
StatusFileSize
new22.17 KB
new947 bytes

This makes QueryArgsCacheContext::getContext() always return a string. This would also fix a problem with query strings like '?value[0]=a&value[1]=b', see #2501905-7: \Drupal\views\Plugin\views\argument_default\QueryParameter should specify a more specific cache context.

Aki Tendo’s picture

Status: Needs review » Reviewed & tested by the community

Verified to work. The contents of the #4 interdiff can be committed to trunk alone if desired - that's the actual fix and it is not reliant on the rest of the patch which is just an assert statement test that discovered the problem.

xjm’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +Needs beta evaluation

Thanks @Aki Tendo and @olli!

The contents of this patch seem radically out of scope for the title of the issue, so let's create a patch that just fixes the bug. A simple unit test or such would be preferable for test coverage.

Also, the issue needs a beta evaluation.

xjm’s picture

Status: Needs review » Needs work
olli’s picture

Status: Needs work » Needs review
StatusFileSize
new2.59 KB
new3.53 KB
new700 bytes

Here's a test.

The last submitted patch, 8: 2509436-8-test.patch, failed testing.

dawehner’s picture

  1. +++ b/core/lib/Drupal/Core/Cache/Context/QueryArgsCacheContext.php
    @@ -28,11 +28,12 @@ public static function getLabel() {
    +      return $query_string === NULL ? '' : $query_string;
    ...
    +    return $value === NULL ? '' : http_build_query([$query_arg => $value], '', '&');
    

    What about just casting that into a string.

  2. +++ b/core/tests/Drupal/Tests/Core/Cache/Context/QueryArgsCacheContextTest.php
    @@ -0,0 +1,64 @@
    +    $container_prophecy = $this->prophesize('Symfony\Component\DependencyInjection\ContainerInterface');
    +    $container_prophecy->get('cache_context.url.query_args')->willReturn(new QueryArgsCacheContext($request_stack));
    

    <3

olli’s picture

StatusFileSize
new3.51 KB
new1.64 KB

Thanks for the review @dawehner. Casting seems to work too.

olli’s picture

StatusFileSize
new3.46 KB
new901 bytes
  1. +++ b/core/lib/Drupal/Core/Cache/Context/QueryArgsCacheContext.php
    @@ -30,11 +30,12 @@ public static function getLabel() {
    +      $query_string = $this->requestStack->getCurrentRequest()->getQueryString();
    +      return (string) $query_string;
    

    removed $query_string variable.

  2. +++ b/core/lib/Drupal/Core/Cache/Context/QueryArgsCacheContext.php
    @@ -30,11 +30,12 @@ public static function getLabel() {
    +    return (string) http_build_query([$query_arg => $value], '', '&');
    

    No need to cast here.

dawehner’s picture

Nice test coverage!

+++ b/core/tests/Drupal/Tests/Core/Cache/Context/QueryArgsCacheContextTest.php
@@ -0,0 +1,64 @@
+    $container_prophecy = $this->prophesize('Symfony\Component\DependencyInjection\ContainerInterface');

Just for future reference, you can now use ::class

Aki Tendo’s picture

StatusFileSize
new2.05 KB
new1.97 KB

With the assert tools patch committed to core I can now append the original failing assertion to the patch.

Note: Due to an issue with the test bots the .htaccess file in the patch has assert active set to 1. This change is not to be included in the final patch.

Aki Tendo’s picture

StatusFileSize
new4.63 KB

K, it passed. Uploading patch with .htaccess file change reverted for final commit.

Aki Tendo’s picture

StatusFileSize
new5.05 KB

As #15 but with assertions back on - the file size differences clued me in that the #14 patch was just the runtime assertion and the fix and didn't hold olli's new unit test code.

Aki Tendo’s picture

Issue summary: View changes
Issue tags: -Needs beta evaluation
Aki Tendo’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Runtime assertion

Ok, I'm going to mark this RTBC since the only code I've wrote in this have all been the test failure that olli later wrote code to pass. It is the patch posted at #15 that needs to go be committed to trunk.

olli’s picture

Status: Reviewed & tested by the community » Needs review
+++ b/core/lib/Drupal/Core/Cache/Context/CacheContextsManager.php
@@ -107,6 +107,8 @@ public function getLabels($include_calculated_cache_contexts = FALSE) {
@@ -124,7 +126,6 @@ public function convertTokensToKeys(array $context_tokens) {

@@ -124,7 +126,6 @@ public function convertTokensToKeys(array $context_tokens) {
       list($context_id, $parameter) = $context;
       $keys[] = $this->getService($context_id)->getContext($parameter);
     }
-
     // Create the returned object and merge in the cacheability metadata.
     $context_cache_keys = new ContextCacheKeys($keys);
     return $context_cache_keys->merge($cacheable_metadata);

@Aki Tendo: Did you mean to add an assert for $keys here? I think that's what failed in #1.

Aki Tendo’s picture

StatusFileSize
new6.4 KB

Sorry it took so long to get back to this. In the code I originally added the assertion to there was no ContextCacheKeys object. So the original test is now outdated.

Submitted is a new test - the check of the validity of the keys now lies in the ContextCacheKeys constructor since it declares it must take an array of strings as an argument. The merge function of Cacheable Metadata asserts that it is returning a new instance of itself. The new QueryArgsCacheContextTest now has 16 failures - possibly cause it also has slipped out of sync with motion in the code base.

Status: Needs review » Needs work

The last submitted patch, 20: failingtest--2509436-20.diff, failed testing.

Aki Tendo’s picture

Well, good news - the only fails are coming out of olli's new test.

olli’s picture

Status: Needs work » Needs review
StatusFileSize
new5.59 KB
new3.04 KB

Fixed the test and reverted .htaccess changes.

Aki Tendo’s picture

Looks great to me, anyone else?

olli’s picture

StatusFileSize
new5.64 KB
new1.24 KB

Just for future reference, you can now use ::class

That is nice, thank you.

Reroll.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Status: Needs review » Needs work

Found this via #2729439: QueryArgsCacheContext should return a special value for ?arg (without value).

  1. +++ b/core/lib/Drupal/Core/Cache/CacheableMetadata.php
    @@ -128,6 +128,8 @@ public function merge(CacheableMetadata $other) {
    +    assert('$result instanceof \\Drupal\\Core\\Cache\\CacheableMetadata');
    

    I think this assertion is quite pointless, because $result = clone $this;, so that's always true.

  2. +++ b/core/lib/Drupal/Core/Cache/Context/CacheContextsManager.php
    @@ -107,6 +107,8 @@ public function getLabels($include_calculated_cache_contexts = FALSE) {
    +    assert('\\Drupal\\Component\\Assertion\\Inspector::assertAllStrings($context_tokens)');
    
    +++ b/core/lib/Drupal/Core/Cache/Context/ContextCacheKeys.php
    @@ -28,6 +28,7 @@ class ContextCacheKeys extends CacheableMetadata {
    +    assert('Drupal\\Component\\Assertion\\Inspector::assertAllStrings($keys)');
    

    These assertions OTOH are very valuable!

  3. +++ b/core/lib/Drupal/Core/Cache/Context/QueryArgsCacheContext.php
    @@ -30,11 +30,11 @@ public static function getLabel() {
    +      return (string) $this->requestStack->getCurrentRequest()->getQueryString();
    

    This is the bugfix the issue summary/title state.

  4. +++ b/core/lib/Drupal/Core/Cache/Context/QueryArgsCacheContext.php
    @@ -30,11 +30,11 @@ public static function getLabel() {
    +    return http_build_query([$query_arg => $value], '', '&');
    

    I don't think this is a good idea. \Drupal\Core\Cache\Context\CacheContextsManager::convertTokensToKeys() is already ensuring a clear key-value pair.

    Let's not change this.

olli’s picture

Thanks for your review!

4. We need that (in addition to 3) to always get a string from QueryArgsCacheContext::getContext(). Without it, both '?value[min]=1' and '?value[min]=2' for 'url.query_args:value' evaluate to 'Array' with a notice 'Array to string conversion' from CacheContextsManager::convertTokensToKeys().

wim leers’s picture

Oh, interesting. Fine, let's do some conversion from array to string, but let's not repeat $query_arg.

Also, #2729439: QueryArgsCacheContext should return a special value for ?arg (without value) just landed, which already added QueryArgsCacheContextTest. That means this patch's test will need to be refactored.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

andypost’s picture

Version: 8.4.x-dev » 8.5.x-dev
Issue tags: +Needs issue summary update
Related issues: +#2738563: Headers cache context does not work, also needs test coverage, +#2744421: QueryArgsCacheContext::getContext() may return NULL, violates the interface

The related issues fixed now it needs work and reroll

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Issue summary: View changes
Issue tags: +Needs reroll, +Bug Smash Initiative

This was a bugsmash daily target. I ran the test that is in the patch and it fails, so I assumed this still needed to be done. larowlan confirmed that in slack.

This also needs an issue summary update, see Write an issue summary for an existing issue for guidance.

quietone’s picture

Title: Drupal\views\Plugin\views\cache\CachePluginBase->generateResultsKey() passes invalid data to CacheContextsManager->convertTokensToKeys » Make CachePluginBase::generateResultsKey() to pass valid data to CacheContextsManager::convertTokensToKeys

Trying to improve the title

bhaveshithape’s picture

Status: Needs work » Needs review
StatusFileSize
new544 bytes

Adding Rerole for d10, As previous patch failed to apply on d10 and also most of the files already had the changes so there is minor change in the patch.

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Regarding #46

also most of the files already had the changes

How so? I'm checking D10.1 and I don't see the changes for a few of those files. Also this lost test coverage so adding that tag back

Was previously tagged for issue summary which still needs to happen as well.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

dimitriskr made their first commit to this issue’s fork.

dimitriskr’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs issue summary update, -Needs reroll, -Needs tests

I've altered the existing test to support the new format brought by $cache_context->getContext()

smustgrave’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: +Needs subsystem maintainer review

Issue summary section for solution is missing.

Tagging for submaintainer to make sure they agree on the approach

dimitriskr’s picture

Issue summary: View changes
Status: Needs work » Needs review
berdir’s picture

Status: Needs review » Needs work

Those asserts are not doing anything, see MR comment.

dimitriskr’s picture

Status: Needs work » Needs review
kristiaanvandeneynde’s picture

Status: Needs review » Needs work

I feel like #27.1 was never addressed. My assumption is also that if CacheableMetadata::merge() has $result = clone $this;, then it seems kind of moot to assert($result instanceof CacheableMetadata);.

Comment #28 also touched on the ampersand but not sure if we really need that.

From a cache submaintainer point of view, I see nothing wrong here. Assertions are harmless, casting casche context return values to strings is too because worst case scenario that leads to a few cache entries no longer being reachable, at which point they get recalculated and stored again. Only been subsystem maintainer for a few months now, so will ask @catch before removing tag.

kristiaanvandeneynde’s picture

This looks fine conceptually, still NW per my latest comment, though.

bhaveshithape’s picture

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.