Problem with render caching of Paragraph-referenced entities [#2855735]

Description:

We are experiencing caching issues when using Paragraphs containing entity references (to nodes in our case).
On the content type that contains a paragraph field, a paragraph could be added that provides an entity reference to nodes of another content type. When saving the paragraph-containing node, everything works as expected (node referenced via paragraph gets rendered, based on the paragraph display settings).

However, when the referenced node gets updated, the paragraph-containing node shows outdated information, when caching is enabled.

Debugging the cache tags, I noticed there is no cache tag on the paragraph-containing node that is linked to the referenced node, so its render cache does not get invalidated when the referencing node would change.

Steps to reproduce:

* Create basic content type (e.g. Page)
* Create Paragraph type, e.g. "Related content", which contains one entity_reference field, that is able to reference nodes of type 'page'.
* Create content type News
* Add Paragraphs field to CT News, allowing paragraphs of type "Related content"
* Configure paragraph display settings to use "Rendered entity" for the entity_reference field
* Enable Drupal core page_cache / dynamic_page_cache module, and make sure a real caching backend is used (e.g. DB caching)
* Create a page node with title "test" and body "version 1"
* Create a news node with a paragraph referencing node "test"
* View news node. It should show "version 1" on the paragraph field output.
* Edit node "test" and change body to "version 2"
* View news node. It still shows "version 1"

Context:

* Using Drupal 8.2.6
* Using latest releases of Paragraphs (1.1) and Entity Reference Revisions (1.2)

I currently fixed this with some custom code that uses hook_entity_build_defaults_alter() to add extra cache tags.
Basically it checks for existing paragraph fields that contain an entity_reference field.
If found, it adds the 'node:[NID]' tag for each referenced node.
Now the last step in the "Steps to reproduce" above contains the right information, instead of the outdated.

I have searched for an existing issue in the queue, but couldn't find one.
Any pointers on how this problem could be fixed (preferably without using custom code), or where patches would need to be applied to fix it, would be very much appreciated.

Comment	File	Size	Author
#24	screenshot_2023-09-12_at_15_31_29.png	98.22 KB	alphex
#19	paragraphs-render_cache-2855735-18.patch	1.11 KB	cgoffin
#19	8.x-1.x: PHP 7.3 & MySQL 5.7, D9.4 160 pass, 12 fail

Issue fork paragraphs-2855735

Show commands

Start within a Git clone of the project using the version control instructions.

Add & fetch this issue fork’s repository

Or, if you do not have SSH keys set up on git.drupalcode.org:

Add & fetch this issue fork’s repository

2855735-problem-with-render changes, plain diff MR !45
Check out this branch

About issue forks

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

24 February 2017 at 15:04

svendecabooter created an issue. See original summary.

Comment #2

miro_dietiker

Switzerland

CreditAttribution: miro_dietiker at MD Systems GmbH commented 24 February 2017 at 15:15

Priority:

Normal

» Major

As i don't like stale-caching issues, i'm promoting this. :-)

Comment #3

svendecabooter

Dutch

Gent

CreditAttribution: svendecabooter at Sven Decabooter for Torchbox, The Chartered Society of Physiotherapy commented 24 February 2017 at 16:11

For reference: here is the code / custom module that fixes this problem: https://github.com/svendecabooter/csp_cache
Doesn't feel like the proper way to handle this, so insights from anyone more familiar with Paragraphs / render caching would be wonderful.

Comment #4

fgm

French

Paris, France

CreditAttribution: fgm at OSInet commented 28 June 2017 at 14:18

Similar but not identicaly problem in the opposite situation (node -> paragraph vs paragraph -> node):

* a node, carrying an EntityReferenceRevision to a Paragraph..
* carrying an ERR to another Paragraph
* carrying an ER to a custom Product entity

Page displaying the node contains tags node:(nid), paragraph:(pid1), paragraph:(pid2), product:(pid)

Update the product: invalidations appear in the cachetags table on the lines for paragraph:(pid1), paragraph:(pid2), and product:(pid) but not on the node:(nid) line. Editing the node contains the proper content and saving it causes the missing invalidation.

Comment #5

Berdir

German

Switzerland

CreditAttribution: Berdir at MD Systems GmbH commented 4 July 2017 at 16:35

Status:

Active

» Postponed (maintainer needs more info)

Are you sure this also happens on a plain D8 site with standard templates and formatters? We use the standard entity reference formatters, cache tags should bubble up through the entities.

Re #4: In case of updating the product, the only cache tag that should be updated should actually just be the product ID, which should be enough to invalidate that other page.

Comment #6

ThomasThe4th CreditAttribution: ThomasThe4th commented 12 July 2017 at 17:44

I am having the same issue. In my case I have a paragraph that contains a node reference. When I look at the cachetags in the response, I see the reference to the paragraph. I do not see a reference to the node, which I understand may be by design. If I update the node and refresh, the paragraph is not invalidated and shows stale info.

Comment #7

Berdir

German

Switzerland

CreditAttribution: Berdir at MD Systems GmbH commented 12 July 2017 at 17:55

How is the node displayed on the paragraph, through a normal formatter or are you doing something custom in template/preprocess?

Comment #8

clittell CreditAttribution: clittell as a volunteer commented 2 November 2017 at 15:02

I am having the same issue.

Comment #9

bdlangton CreditAttribution: bdlangton commented 2 January 2018 at 19:31

I'm not sure for ThomasThe4th, but I had this issue too, but found that the issue is from the entity_reference_tab_formatter module. Somebody has suggested a patch over there: https://www.drupal.org/project/entity_ref_tab_formatter/issues/2923378

As far as I can tell, there isn't an issue with Paragraphs itself.

Comment #10

markus_petrux CreditAttribution: markus_petrux commented 29 April 2019 at 13:29

Hi!

I've have faced a use case where this issue is happening to us.

We have a paragraph with a node reference that is visible to a user depending on a field on the paragraph.

- When the node is rendered (because the user is allowed to see it), the cache tag is generated and the cached rendered array is expired when the node referred is updated. This is ok.

- When the node is not rendered (because the user is NOT allowed to see it), teh cache tag is NOT generated, so the cached rendered array is not affected when the node is updated. This is not correct.

The cached rendered array should be expired, because the updated node could be visible to the user after the change.

In our use case, we have to include all cache tags, even those that are part of render array thay is not visible.

Not sure how to do it yet... but I think it may shed some light on the issue...

Cheers!

Comment #11

Berdir

German

Switzerland

CreditAttribution: Berdir at MD Systems GmbH commented 29 April 2019 at 13:33

Does that *only* happen when you have a node reference in a paragraph or also if you have such a node reference field directly on an node? If the second, then it's not a paragraph issue but a general drupal core issue. I know there are some open issues around cache tags sometimes not being merged correctly.

Comment #12

markus_petrux CreditAttribution: markus_petrux commented 29 April 2019 at 14:03

Yes, it seems something beyond Paragraph module. ie. seomthing that happens somewhere inside core's EntityReferenceFormatterBase::getEntitiesToView(), which is what I'm looking into now...

Comment #13

markus_petrux CreditAttribution: markus_petrux commented 29 April 2019 at 15:48

Well, for the moment I fixed my use case generating cache tags for all referred nodes, despite they are accessible or not. That way, these cached entries are always expired when the referred nodes are updated.

Any idea on which core issue could this be reported? (I've tried to search with no luck)

Thanks!

Comment #14

Elijah Lynn

Portland, Oregon

CreditAttribution: Elijah Lynn at Agile Six Applications for Department of Veterans Affairs commented 2 October 2019 at 18:49

Woah, I just arrived at this issue too after an hour or so of debugging. I have a block using references to paragraphs and has the wrong data in the display but the right data in an SQL query from the CLI. If I cache:rebuild nothing happens, including manually truncating cache tables. If I edit the node and save my change to the old data that is displaying, it overwrites the correct value in the DB. This is from a synced DB. I bet it is mysql query cache maybe. I'll post more back when I figure it out.

Comment #15

Elijah Lynn

Portland, Oregon

CreditAttribution: Elijah Lynn at Agile Six Applications for Department of Veterans Affairs commented 2 October 2019 at 20:02

In our case it seems to just have been a coincidence. The old content we were seeing wasn't old but it was drupal 8 native filter protocols stripping the SMS on our CI environment, which wasn't getting our services.yml loaded that had SMS protocol whitelisted. That was crazy debugging!

Comment #16

ryrye CreditAttribution: ryrye commented 9 September 2020 at 20:05

Thank you @svendecabooter your code at https://github.com/svendecabooter/csp_cache (mentioned in #3) fixed this problem for me.

This appears to still be an issue as of now. svendecabooter's code should probably be incorporated into the paragraphs module as a permanent fix.

Comment #17

28 October 2022 at 14:29

cgoffin made their first commit to this issue’s fork.

Comment #18

28 October 2022 at 14:31

cgoffin opened merge request !45

Comment #19

cgoffin CreditAttribution: cgoffin at Sopra Steria commented 28 October 2022 at 14:41

File	Size
paragraphs-render_cache-2855735-18.patch	1.11 KB
8.x-1.x: PHP 7.3 & MySQL 5.7, D9.4 160 pass, 12 fail

I have a similar problem with paragraphs in combination with the viewsreference module. The viewsreference module allows us to link to a view within a paragraph. This works fine. But when someone goes for example to /paragraph-page?page=9999 and there is no page 9999 the page gets cached without the url.query_args context and when you then go to /paragraph-page you also get an empty view while there are results for the view. To fix this I created a patch to bubble up caching information from the paragraph elements onto the paragraph.

I think this will also fix the problem discussed in this ticket and the fix of #3 will not be necessary anymore. Maybe someone else can test this use case.