Problem/Motivation

Enabling the new Navigation module in Drupal 11.3.2 (and likely earlier) results in node updates not being immediately reflected in the rendered nodes. Clearing the cache for that node or globally updates the rendered output, making it clear this is a caching issue. Disabling Navigation results in normal behavior.

Steps to reproduce

Install the Navigation module.
Edit a node's body or a similar field and save.

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3566843

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

  • 3566843-navigation-module-results Comparecompare
    • 1 hidden branch
  • main Comparecompare
About issue forks

Comments

rozzi created an issue. See original summary.

nicxvan’s picture

nicxvan commented

Can you provide any more details?

rozzi’s picture

rozzi commented

For our website, the two steps are enough to reproduce the issue every time, on nodes that are blogs and that aren't blogs. I'm running Drupal 11.3.2, but I'm pretty sure the issue was present in 11.3.1 as well, if not earlier. Is there anything specific you'd like me to try or note?

rozzi’s picture

rozzi commented
Issue summary: View changes
nicxvan’s picture

nicxvan commented

Are you using a cdn?
Does this happen locally with ddev?

Edit:
What blocks do you have in the navigation? Just defaults?

Also please keep the template in the issue summary, it's helpful for core maintainers.
If you don't have information for one then you can leave the section blank.

quietone’s picture

quietone commented
Version: 11.3.x-dev » 11.x-dev

Hi, in Drupal core changes are made on on 11.x (our main development branch) first, and are then back ported as needed according to the Core change policies. The version the problem was discovered should be stated in the issue summary Problem/Motivation section. Thanks.

@nicxvan, thanks for reminder to retain the issue summary! Although, I would add that it is helpful to all contributors, particularly for reviewers so they can easily get up to speed on an issue.

rozzi’s picture

rozzi commented
Issue summary: View changes
rozzi’s picture

rozzi commented

I'm not entirely sure about the Drupal versions affected, as I enabled Navigation as advised in the Drupal 11.3 blog post after updating to 11.3. We use Cloudflare, but since disabling it has no effect and nodes get updated from clearing the Drupal cache or uninstalling Navigation, I'm quite sure the issue is isolated to the Navigation module. Installing and uninstalling it consistently reintroduces or eliminates the issue. Its only settings are related to its logo, so I'm not aware of any custom blocks at play.

rozzi’s picture

rozzi commented
Issue summary: View changes
catch’s picture

catch
he/him
Primary language English
commented

Are you able to reproduce this with a clean install of Drupal core? Could use either standard or Umami with toolbar uninstalled and navigation enabled.

godotislate’s picture

godotislate
he/him
commented

Tested locally per #10 with clean install of 11.3.2 core with both Standard and Umami profiles, with toolbar uninstalled and navigation installed. I did not set up any CDN integration, so the outermost caching layer is the Drupal page cache.

After I created an Article in either profile and then viewed it anonymously in a private window, the content appears as expected. If I edit and save changes to the Article body, and then refresh the private window, the content appears update as expected, so if this is a navigation or core-specific issue, it probably has to do with interaction of navigation with contrib or custom modules and themes.

One thing I did notice with the cache debugging headers turned on is that when viewing an Umami article anonymously, there are more cache tags when navigation is installed, specifically these:

  • config:core.entity_view_display.block_content.footer_promo_block.default
  • config:core.entity_view_display.media.image.medium_8_7
  • config:core.entity_view_display.media.image.responsive_3x2
  • config:core.entity_view_display.node.article.card

All other tags are the same when navigation is installed vs toolbar being installed. I haven't looked into why the tags are added, but it does bring up one possibility if the cache invalidation issue is at the Cloudflare/CDN level: the additional cache tags might cause the purge request exceed the limit on the number cache tags that Cloudflare allows to be invalidated in a request, or something similar.

@rozzi:
A couple things to try:

  • With navigation enabled, after you update content and view it anonymously, if you add some sort of cache breaking query string (for example ?{any string of pretty random characters}) to the URL, does it appear correctly?
  • If you have a secondary domain for the same site that does not go through the CDN, are you seeing the same behavior when the page is viewed using that domain?
  • Are all content types affected? Or only certain ones?
rozzi’s picture

rozzi commented

Thanks, godotislate. I didn't have the prerequisites for a local install, so I thought it would be much quicker and easier for someone to test on their own website, even if it weren't a fresh install. I was hoping to go from there.

If you have a secondary domain for the same site that does not go through the CDN, are you seeing the same behavior when the page is viewed using that domain?

I switch Cloudflare to the Development mode when testing this, but as noted earlier, the issue is completely unaffected by that but consistently affected by clearing the Drupal cache either via Performance, manually for the test node(s) via a drush command, or by disabling Navigation. The latter immediately fixes the problems.

To comment on your mention of themes and custom modules, switching to a base theme or clearing the .module of our only custom module and refreshing the cache doesn't help either. I'd rather not risk uninstalling the contrib modules in production.

With navigation enabled, after you update content and view it anonymously, if you add some sort of cache breaking query string (for example ?{any string of pretty random characters}) to the URL, does it appear correctly?

It doesn't look like it. When I access an updated page in an incognito window for the first time, it shows its actual content, even when I don't see it in the main window yet. If I update it, the updates aren't visible on either end, and adding ?1 for example doesn't help.

Are all content types affected? Or only certain ones?

All of them it seems. I've tested across four types of nodes and a user profile. Moreover, if I have a view inside a block set to display the latest nodes and I remove a node, it's the cached version that shows. I haven't tested this last part extensively though, so we can limit the scope of this to nodes. It's clear the issue is related to caching, so solving one would most likely solve the other.

nicxvan’s picture

nicxvan commented

Can you provide a list of modules you have enabled from contrib?

rozzi’s picture

rozzi commented

Sure:

 --------------------------------- ------------------------------------------------------ --------- ---------------
  Package                           Name                                                   Status    Version
 --------------------------------- ------------------------------------------------------ --------- ---------------
  Other                             Advanced Queue (advancedqueue)                         Enabled   8.x-1.5
  Other                             Backup and Migrate (backup_migrate)                    Enabled   5.1.1
  Other                             Better Formats (better_formats)                        Enabled   8.x-1.0-beta3
  Other                             Blog (blog)                                            Enabled   3.1.4
  Modeler API                       BPMN.iO Modeler (bpmn_io)                              Enabled   3.0.4
  Spam control                      CAPTCHA (captcha)                                      Enabled   2.0.9
  Spam control                      Image CAPTCHA (image_captcha)                          Enabled   2.0.9
  User interface                    Chosen Library (chosen_lib)                            Enabled   5.0.3
  Media                             Colorbox (colorbox)                                    Enabled   2.1.4
  Other                             Configuration Update Base (config_update)              Enabled   2.0.0-alpha4
  Other                             Configuration Update Reports (config_update_ui)        Enabled   2.0.0-alpha4
  Web services                      Serialization (CSV) (csv_serialization)                Enabled   4.0.1
  Chaos tool suite                  Chaos Tools (ctools)                                   Enabled   4.1.0
  Chaos tool suite (Experimental)   Chaos Tools Blocks (ctools_block)                      Enabled   4.1.0
  Other                             Entity Mask (ctools_entity_mask)                       Enabled   4.1.0
  Chaos tool suite (Experimental)   Chaos Tools Views (ctools_views)                       Enabled   4.1.0
  DANSE                             Drupal Audit Notification Subscription Event (danse)   Enabled   2.3.6
  DANSE                             DANSE Content (danse_content)                          Enabled   2.3.6
  Development                       Devel (devel)                                          Enabled   5.5.0
  Development                       Devel Generate (devel_generate)                        Enabled   5.5.0
  Other                             Diff (diff)                                            Enabled   2.0.0-beta4
  ECA                               ECA Core (eca)                                         Enabled   3.0.9
  ECA                               ECA Base (eca_base)                                    Enabled   3.0.9
  ECA                               ECA Content (eca_content)                              Enabled   3.0.9
  ECA                               ECA UI (eca_ui)                                        Enabled   3.0.9
  ECA                               ECA User (eca_user)                                    Enabled   3.0.9
  ECA                               ECA Views (eca_views)                                  Enabled   3.0.9
  Views                             Eva (eva)                                              Enabled   3.1.1
  Migration                         Field Group Migrate (field_group_migrate)              Enabled   4.0.0
  Fields                            Field Group (field_group)                              Enabled   4.0.0
  Fields                            Field Permissions (field_permissions)                  Enabled   8.x-1.5
  Fields                            File (Field) Paths (filefield_paths)                   Enabled   8.x-1.0-rc1
  Other                             Gin Toolbar (gin_toolbar)                              Enabled   3.0.2
  Forms                             Hide Revision Field (hide_revision_field)              Enabled   8.x-2.5
  Image                             Image Field Tokens (imagefield_tokens)                 Enabled   3.2.0
  Media                             Insert (insert)                                        Enabled
  Media                             Insert Colorbox (insert_colorbox)                      Enabled
  Other                             Libraries (libraries)                                  Enabled   4.1.1
  Mail                              Mail System (mailsystem)                               Enabled   8.x-4.5
  SEO                               Metatag (metatag)                                      Enabled   2.2.0
  SEO                               Metatag: Mobile & UI Adjustments (metatag_mobile)      Enabled   2.2.0
  SEO                               Metatag: Open Graph (metatag_open_graph)               Enabled   2.2.0
  SEO                               Metatag: Twitter Cards (metatag_twitter_cards)         Enabled   2.2.0
  SEO                               Metatag: Verification (metatag_verification)           Enabled   2.2.0
  Migrate                           Migrate Devel (migrate_devel)                          Enabled   3.0.0
  Migration                         Migrate Plus (migrate_plus)                            Enabled   6.0.9
  Migration                         Migrate Tools (migrate_tools)                          Enabled   6.1.3
  Migration                         Migrate Upgrade (migrate_upgrade)                      Enabled   4.0.2
  Modeler API                       Modeler API (modeler_api)                              Enabled   1.0.6
  Other                             Pathauto (pathauto)                                    Enabled   8.x-1.14
  Input filters                     Pathologic (pathologic)                                Enabled   2.0.0-alpha3
  Push                              Push Framework Email (pf_email)                        Enabled   2.3.0
  Media                             Plupload Integration Module (plupload)                 Enabled   2.2.0
  Push                              Push Framework (push_framework)                        Enabled   2.3.6
  Spam control                      reCAPTCHA (recaptcha)                                  Enabled   8.x-3.4
  Other                             Redirect 404 (redirect_404)                            Enabled   8.x-1.12
  Other                             Redirect (redirect)                                    Enabled   8.x-1.12
  User interface                    Simple Hierarchical Select (shs)                       Enabled   3.0.0-beta5
  Statistics                        Statistics (statistics)                                Enabled   1.0.0
  Other                             Sub-pathauto (subpathauto)                             Enabled   8.x-1.4
  User interface                    Superfish (superfish)                                  Enabled   8.x-1.13
  Other                             Term Merge (term_merge)                                Enabled   2.0.0
  Other                             Term Reference Change (term_reference_change)          Enabled   2.0.0-beta5
  Other                             Token (token)                                          Enabled   8.x-1.17
  Other                             Typed Data (typed_data)                                Enabled   2.1.1
  Views                             Views Data Export (views_data_export)                  Enabled   8.x-1.8
  Views                             Views Slideshow Cycle (views_slideshow_cycle)          Enabled   5.0.1
  Views                             Views Slideshow (views_slideshow)                      Enabled   5.0.1
  Extensible BBCode                 Standard tags (xbbcode_standard)                       Enabled
  Extensible BBCode                 Extensible BBCode (xbbcode)                            Enabled
                                    Gin (gin)                                              Enabled
                                    Rivet (rivet)                                          Enabled

The enabled Captcha points are contact_message_personal_form and user_register_form. When I had Navigation enabled, Gin Toolbar was disabled, though keeping it enabled doesn't change the behavior.

godotislate’s picture

godotislate
he/him
commented

@rozzi
There doesn't seem to be any Purge or Cloudflare modules in that list. Does that mean you are not using Cloudflare as a caching layer (i.e. no caching, but only DNS or WAF)? And presumably no caching layer between Cloudlfare and Drupal?

If so, we can probably try to isolate what Drupal layer the caching issue is occurring at with some testing. With navigation installed and toolbar uninstalled:

  1. Uninstall the Internal Page Cache, and check to see if content is updating correctly.
  2. If content is still not update correctly, uninstall the Dynamic Page Cache, and check to see if content is updating correctly
rozzi’s picture

rozzi commented

There is no integration of Cloudflare with Drupal on our end indeed. However, we still do some caching via Cloudflare externally. Of anything that affects the site globally rather than its specific paths unrelated to nodes, there's just this:

(http.request.uri.path.extension in {"7z" "avi" "avif" "apk" "bin" "bmp" "bz2" "class" "css" "csv" "doc" "docx" "dmg" "ejs" "eot" "eps" "exe" "flac" "gif" "gz" "ico" "iso" "jar" "jpg" "jpeg" "js" "mid" "midi" "mkv" "mp3" "mp4" "ogg" "otf" "pdf" "pict" "pls" "png" "ppt" "pptx" "ps" "rar" "svg" "svgz" "swf" "tar" "tif" "tiff" "ttf" "webm" "webp" "woff" "woff2" "xls" "xlsx" "zip" "zst"})

(with the theme CSS being exempt via a separate rule)

Uninstalling the two cache extensions you suggested still doesn't solve the problem though.

catch’s picture

catch
he/him
Primary language English
commented

@rozzi there are two other places the node gets cached.

It looks like you're probably using database caching (can't see redis in the module list you posted).

Can you try the following - typing this from memory so the instructions might not be perfect:

TRUNCATE cache_render

Then visit the page you're using to test (only that page).

SELECT cid, created FROM cache_render
- try to find the render cache item for the node you're viewing and make a note of the created time.

Then edit the node again.

Then
SELECT cid, created FROM cache_render
And see if the cache item you found earlier has a new timestamp.

And then assuming the node hasn't updated, try truncating cache_render again, and visiting the node, and see if it updates then.

If it's the cache_render entry that's the problem, then we'll need to narrow it down further, e.g.:

1. Is it getting invalidated at all when the node is saved
2. If so, is it somehow rendering with the old node values.

You can also do a similar test again with the cache_entity table for the node that you're updating, that's much less likely to go wrong but it's not impossible if a module is doing something very strange within node save.

None of this explains the interaction with the navigation module but it might help us get there.

rozzi’s picture

rozzi commented

Hi @catch, I'm not getting consistent results from those queries. Without Navigation, it seems that going to the node page for the first time or editing it immediately updates the timestamps. With Navigation, I remember truncating the table having an effect, however, doing it now and visiting or editing the page does not create any rows in the table.

catch’s picture

catch
he/him
Primary language English
commented

however, doing it now and visiting or editing the page does not create any rows in the table.

This will be the case if dynamic page cache and internal page cache are enabled and have warm caches, because the individual node won't get rendered at all in that case.

rozzi’s picture

rozzi commented

With those two caches disabled again, I saw no updates (or timestamp changes) from editing the node with Navigation enabled, but truncating the render cache updated the node both times I tried.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

hongqing’s picture

hongqing commented

I have the cache problem with navigation. Other users can see the Navigation.

Drupal: 11.3.1
Cache: AWS ElastiCache
Clouflare

I find the problem is caused by cloudflare cache. It works after bypass everything, but it does not after bypass uri path start with /admin or url path start with /user. This is resovled by setting TTL to 0 in cloudflare.

mark_fullmer made their first commit to this issue’s fork.

mark_fullmer’s picture

mark_fullmer
he/him
Primary language English
Location Tucson
commented

I was able to reproduce this problem on clean Drupal install with just the Redirect contrib module added.

I was able to confirm that the problem only occurs when both the Navigation module and the Redirect module are enabled, and does not occur if I comment out the caching logic in NavigationRenderer::doBuildNavigation(), or if I perform a cache flush.

I've created an issue in #3584029: Cache conflict with Drupal Navigation module, along with four FunctionalJavascript test scenarios that demonstrate the problem only occurs when both Navigation and Redirect are enabled.

    Navigation Cache (Drupal\Tests\redirect\FunctionalJavascript\NavigationCache)
     ⚠ Passing cache flush
     ⚠ Passing no navigation
     ⚠ Passing no redirect
     ✘ Failing navigation and redirect
       ┐
       ├ Behat\Mink\Exception\ExpectationException: Current page is "/node/1", but "/test-page" expected.

I note that the original reporter of this issue encountered the problem on a site where Redirect was enabled, too, per https://www.drupal.org/project/drupal/issues/3566843#comment-16416217

bluegeek9 made their first commit to this issue’s fork.

bluegeek9 changed the visibility of the branch main to hidden.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented

Based on my research in the linked redirect issue, this is a race condition that might fix itself with #3585957: Prevent the navigation sidebar being rendered via big pipe as navigation will no longer be rendered after sending the response.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented
Status: Active » Postponed (maintainer needs more info)

That other issue landed in main and 11.x, please report if you can still report any issues now.

rozzi’s picture

rozzi commented

Updating to Drupal 11.4.9 and doing cr as always did not address the issue for me.