Problem/Motivation

@yched in #2429257-64: Bubble cache contexts:

Also, the 1-4 bullet points after "We have to instill the following thinking steps in D8 developers" are super valuable. Where in our docs can we print it in bold flaming letters ?

This issue is for doing exactly that.

For completeness, these are those bullets:

We have to instill the following thinking steps in D8 developers:

  1. I'm rendering something. That means I need to think of cacheability!
  2. Is this something that's expensive, and therefore is worth caching? If so: what identifies this thing? Those are the cache keys.
  3. Does the representation of the thing I'm rendering vary per role (needs permissions), per URL (needs something of the request), per language, per… something? Those are the cache contexts. (For every single thing that it varies by, I need to specify the cache context to use, so Drupal can apply smart caching.)
  4. When does the representation of the thing I'm rendering become outdated? I.e. which things does it depend upon, so that when those things change, so should my representation? Those are the cache tags.

Proposed resolution

Find the right place to document this, and then iterate on the exact documentation; it must be crystal clear.

Remaining tasks

TBD

User interface changes

None.

API changes

None.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers’s picture

Title: Document the cacheability considerations when generating render arrays » The Cache Trifecta: document the 3 cacheability considerations when generating render arrays
Status: Active » Needs review
FileSize
1.77 KB

On February 20 (1.5 week ago), I actually already began doing this. I updated core.api.php with better cache docs, only to realize that that's the wrong place. From a chat with @Fabianx:

11:01:24 WimLeers: While rerolling that patch, I started updating docs too, because part of this is consolidating the APIs, so we should also consolidate the docs.
11:01:27 WimLeers: So looking at https://api.drupal.org/api/drupal/core%21modules%21system%21core.api.php/group/cache/8
11:01:36 WimLeers: … that specifically recommends using CIDs
11:01:39 WimLeers: doesn't even mention cache keys
11:01:45 WimLeers: so I started fixing that
11:01:56 WimLeers: … only to realize that that is moronic, because this is actually about CacheBackendInterface
11:02:10 WimLeers: … which is blissfully unaware about cache keys/contexts

… so it actually belongs on the "Render API" docs (we don't have a component for that in the issue queue, weird enough, so "base system" it is). Those docs live in theme.api.php, and can be found on a.d.o at https://api.drupal.org/api/drupal/core%21modules%21system%21theme.api.ph....

Attached is the initial patch I wrote back then.


I'd like to hear from you which docs you prefer: the one in the IS, or the one in this patch, or a mix?

larowlan’s picture

Priority: Normal » Major

Because contributed modules that do this wrong will undo all the ground work in core

Wim Leers’s picture

Because contributed modules that do this wrong will undo all the ground work in core

Agreed.

larowlan’s picture

  1. +++ b/core/modules/system/core.api.php
    @@ -372,9 +372,25 @@
    + * The most important thing to know and remember while caching things is the
    

    I think we need examples of each of the three here - an example aids clarity.

  2. +++ b/core/modules/system/core.api.php
    @@ -372,9 +372,25 @@
    + * 2. cache contexts: an array of contexts that indicate the context in which
    + *    the representation of the thing (see 1.) is being viewed. Simply put: the
    + *    state of the viewer (which user, which language, which country …) that
    + *    requires a specific variation of the thing.
    

    The thing that drove this home for me a.k.a. the Eureka moment, was someone somewhere telling me these were analogous to 'Vary' headers - should we add that here as it's a valid point of reference.

yched’s picture

The thing that drove this home for me a.k.a. the Eureka moment, was someone somewhere telling me these were analogous to 'Vary' headers

+ a lot to that. TBH, not sure why we named that "context" rather than "vary" :-)

Wim Leers’s picture

+ a lot to that. TBH, not sure why we named that "context" rather than "vary" :-)

Because you can say "this render array depends on these cache tags", and "this render array depends on these cache contexts", but not "this render array depends on these varies" (nor does "variations" work).


We used to call this "cache granularities" in D7, but there were only a few hardcoded ones (user/role/page/global) and they only existed for blocks. But granularities implies that there's a hierarchy between all of them, and that's also not true.

So we settled on "contexts". This was added in #2158003: Remove Block Cache API in favor of blocks returning #cache with cache tags.


This allows us to nicely explain it: the specific render array variation/representation of something varies by the context it is viewed in — which captures the HTTP Vary header and the cache context abstraction in one clear sentence.

yched’s picture

Yeah, agreed that "vary" makes it harder to make actual sentences with :-/. "Vary context" ?

Berdir’s picture

+++ b/core/modules/system/core.api.php
@@ -372,9 +372,25 @@
  * - Request a cache object through \Drupal::cache() or by injecting a cache
  *   service.
+ * - Define cache keys
  * - Define a Cache ID (cid) value for your data. A cid is a string, which must
  *   contain enough information to uniquely identify the data. For example, if

I'm still confused by cache keys vs cache ID (I think keys should really by cid_parts or something) but this doesn't make sense. AFAIK, having a cid will ignore any cache keys, so you need to have either one and usually cache keys.

Anyway, sounds like this line is cut of/incomplete anyway..

I recently saw something that I wanted to comment here about, something that would be important to document here as well, but I can't remember... :(

Wim Leers’s picture

FileSize
4.75 KB

Let's get this going.

Fabianx’s picture

+++ b/core/modules/system/theme.api.php
@@ -287,6 +287,74 @@
+ *   regenerated. So my cache tags would be: ['node:5', 'user:3', 'file:4',
+ *   'config:filter.format.basic_html'].

This currently seems to imply that I need to set them all myself.

I think its important to distinguish when to add what in another example.

--

Besides this, this is good to go :).

Wim Leers’s picture

FileSize
6.92 KB
3.16 KB

Done.

Fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC, These are great examples!

aspilicious’s picture

- * $build['#attached']['drupalSettings']['foo] = 'bar';
+ * $build['#attached']['drupalSettings']['foo'] = 'bar';

I didn't see this sneaking in...

Wim Leers’s picture

That is just a tiny bug fix in the same area. Or are you talking about drupalSettings in general?

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/system/theme.api.php
    @@ -287,6 +287,134 @@
    + * Since the contents of a render array determine what is shown to the user, it
    + * also depends on render arrays "how cacheable" a page is. If your code is
    

    Does how cacheable need to be in quotes. Also I think the first sentence reads better like Render arrays determine what is shown to the user. Therefore, render arrays also determine how cacheable a response is. Also it might not be "your" code.

  2. +++ b/core/modules/system/theme.api.php
    @@ -287,6 +287,134 @@
    + *    representation of the thing I'm rendering?
    + *    Those are the cache keys.
    + *    They're only necessary if I want to cache this render array.
    ...
    + *    so should my representation?
    + *    Those are the cache tags.
    + *    They're always necessary, because they affect the cacheability of the
    

    Formatting is a bit weird here - seems to be deliberately not flowing to 80 chars. Is this for readability?

  3. +++ b/core/modules/system/theme.api.php
    @@ -287,6 +287,134 @@
    + * - cache keys identify a representation of a thing
    + *   - may only be set if this render array should be cached
    + *   - will trigger caching of this render array (subtree)
    

    Is it worth having an example where cache keys are set?

jhodgdon’s picture

Component: base system » documentation
Status: Needs work » Needs review
FileSize
4.13 KB

@alexpott asked me to take a look at this patch. It seems kind of OK, but it's overly wordy and has way to many Exclamation Points!!! Let's Not Use Those In Docs!

So, I rewrote it. I didn't bother with an interdiff file, since it would be twice as big as the new patch.

Wim Leers’s picture

A huge amount of the information is lost in #16. And, even worse, it contains factually wrong information.

I'm rerolling #11 now, to address #15, reduce wordiness (#16) and remove exclamation points (#16).

Wim Leers’s picture

Title: The Cache Trifecta: document the 3 cacheability considerations when generating render arrays » Document cacheability of render arrays, and the considerations to use when generating render arrays
FileSize
6.76 KB
6.13 KB

Per #17. Interdiff relative to #11.

(Since #11 was RTBC'd, #2443073: Add #cache[max-age] to disable caching and bubble the max-age also landed; so updating this to include that as well.)

jhodgdon’s picture

Status: Needs review » Needs work

Well, this still needs a rewrite. I can't review it today, sorry.

jhodgdon’s picture

OK, here's a quick review... Basically, the proposed documentation in the latest patch is too wordy and too conversational and not very documentation-like.... Put more information on drupal.org if you want and link there, and be conversational in a blog post, but this is meant to be a short overview in a straightforward documentation style. Some specific suggestions:

a) No blank line after the @section line.

b) This is documentation that is going inside the Theme and Render System topic on api.drupal.org. These topics need to be concise and not repeat each other. So, this documentation should not:
- Explain what caching is or why it is desirable. Link to the Cache topic instead.
- Repeat itself at all. No "In other words" paragraphs. Just state the facts once.
- Talk about "importance" of doing stuff or "must". Just state what developers should do in a clear and concise manner.
- Give more than one example. One is enough. Make a page on drupal.org if you want to do more, and link there if necessary, but this is way too much for this topic.

c) The whole thing is very "conversational". We want it to match the style of the rest of the documentation in the topics. And you still have an ! in the text. Please remove. As another example, the first "step" in the process is: "I'm rendering something". This is again the wrong style.

So... that's just a quick review... I think it would be better if you could go back to my patch and fix the factual error (you didn't point out what that was?). My patch is in the style and level of detail that we want to have in these topics, and the information removal was intentional.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
3.92 KB
3.48 KB

(Sister comment of #2373741-9: Move bulk of render pipeline documentation to d.o handbooks.)

Thank you so much for laying out what is expected for api.d.o (*.api.php) docs. Together with #2373741-8: Move bulk of render pipeline documentation to d.o handbooks, I now have a much better understanding of what is supposed to go where. Any chance you could create some docs about docs (:P) to clarify this for everybody? Then you also don't have to keep explaining it, you could just point to that link :)

I've moved my wordy examples into https://www.drupal.org/developing/api/8/render/arrays/cacheability.

As requested, I went back to your patch and fixed the factual errors. I also left out the example, since it's now in the handbook, and on its own it's less useful — the contrast with the other example is necessary for clarity.

P.S.: It'd be better if we'd replace "hierarchy" with "tree" and "portions"/"sections" with "subtrees". But I suspect you find those words too complex/not in line with the rest of the docs?

jhodgdon’s picture

Status: Needs review » Needs work

Great, thanks! I'm fine with tree/subtree. Please go ahead and make that change.

Regarding style docs, see:
https://www.drupal.org/style-guide/content

Patch is looking good, and the drupal.org page is great! A couple of nitpicks in the patch:

a)

+ * this property, always supply the cache contexts, tags and max-age. Cache keys

Serial comma needed in "... tags, and max-age". Appears again later in the paragraph too.

b) Actually, in place of "max-age" can we spell out "maximum age"? Unless "max-age" is what you'd actually use as an array key... what exactly is the array key for this anyway? Maybe it would help if we at least had an example of a #context array you could supply? I think that might be considered part of the "minimal" information that documentation on api.drupal.org should have.

c) I think we need to have something more about cache keys here... or maybe just point out that this is documented on the Cache API page. Just saying they are "identifiers" seems a bit too vague. Again, a concrete example could help people understand what they need to do.

e) Regarding the maximum age, the docs say you should "always" supply it. Is that really true, given that there is a default?

+ * Cache information is provided in the #cache property in a render array. In
+ * this property, always supply the cache contexts, tags and max-age. Cache keys
Wim Leers’s picture

Status: Needs work » Needs review
FileSize
4.33 KB
1.53 KB
  • a) done.
  • b) Unless "max-age" is what you'd actually use as an array key... what exactly is the array key for this anyway? , yep this is the key. Added the requested example render array.
  • c) Does the example added for b) address this?
  • d) … no d? :P
  • e) Tried to clarify that.
jhodgdon’s picture

Status: Needs review » Needs work

Great, this is really looking good, thanks!

Very minor nitpicks now:

a)

+ * render array varies by context, depends on some modifiable data or depends
+ * on information that's only valid for a limited time, respectively. Cache keys

needs a serial comma before "or" here.

b)

+ * The cache contexts, tags and max-age will be propagated up the render array

and here needs a serial comma before "and".

c)

+ * @code
+ * $build = [
+ *   …,

This needs a prefixing line, and probably we can omit the ... part. How about just saying:

 * Here's an example of what a #cache property might contain:
 * @code
 *   '#cache' => [
 *     'keys' => ['entity_view', 'node', $node->id()],

(etc.)

That's it -- rest is very good!

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
4.36 KB
1.55 KB

All done!

P.S.: sorry about repeatedly forgetting the serial comma. I'm afraid that'll take a while for me to notice myself.

jhodgdon’s picture

Status: Needs review » Reviewed & tested by the community

No problem -- you're not alone on the serial comma.

This looks good to me, thanks!

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Docs are frozen in beta. Committed c428646 and pushed to 8.0.x. Thanks!

  • alexpott committed c428646 on 8.0.x
    Issue #2444211 by Wim Leers, jhodgdon: Document cacheability of render...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.