Hello @samuel,

First, thanks for your awesome module, it's really a great project! :)

We are experiencing an issue according to the cache rebuild function. After executing it, as the "cache_tome_static" table is truncated, the whole website is re-exported.
Were you aware about this issue?

Thanks for your answer

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Heisen-blue created an issue. See original summary.

samuel.mortenson’s picture

Category: Bug report » Support request

Hi @Heisen-blue - thanks for the issue.

This is expected functionality for the project, when you rebuild cache you're telling Drupal that all pages on the site need to be regenerated - a cache rebuild will truncate all cache tables as well as invalidate all cache tags. Once this is done, there's no way for Tome to tell if pages will be generated with the same content as the previous build.

Heisen-blue’s picture

Hi @samuel.mortenson

Thanks for your quick reply.

Yes I agree, that makes sense.
IMHO, it doesn't seem logical (and it is a performance issue as well) that the whole website needs to be regenerated after a deployment, or because I enabled a new module and clear cache just after. The website will handle 20000 pages, content editors won't understand why they have to wait the whole re-export each time the cache is rebuild.
Do you think it is a use case we can handle in Tome? Or maybe you have a workaround in mind?

samuel.mortenson’s picture

@Heisen-blue In the short term, I would try to find ways to selectively clear cache without rebuilding the entire cache. For Tome, the cache_tome_static and cachetags tables are what determines if pages get built. Maybe you could truncate all tables except cache_tome_static, and not invalidate cache tags? If this works out we could maybe write a custom Drush command in Tome Static that does this.

For a long term solution, we (or you in a custom module using a service decorator) would need to change the cache.tome_static service to not use cache tags, which seems complex. How can you tell if a page in Drupal needs to be rebuilt without any cache data? I don't really know what this would look like.

Edit: I just realized drush cr doesn't invalidate every cache tag, which is interesting. It does invalidate generic tags like http_response which are present on every page.

samuel.mortenson’s picture

I opened #3026915: New sub-module - Tome Static Watch as another option for making rebuilds a bit less painful.

rozh’s picture

FileSize
21.78 KB

Are you aware of Build Hooks module? It triggers build pipelines on external services like Circle CI. I feel like Tome and Build Hooks have some overlapping features. E.g. Build Hooks tracks changes for selected entity types and shows changelog before deployment.

Build hooks screenshot

In some cases when Tome and statically generated files are hosted on separate servers it is very convenient to have changelog and see what content is going to be rebuild.

I think Build Hooks collect entity changes for informational purposes only. But Tome can use same kind of data to rebuild only changed content pages (and probably related content listing pages).

samuel.mortenson’s picture

Assigned: Unassigned » samuel.mortenson

@rozh I have seen that module, and think that #3026915: New sub-module - Tome Static Watch is eventually going to add similar behavior in a one-off sub-module or another contrib module.

I think what makes this issue difficult to tackle is that Tome's caching and how it chooses pages to re-generate right now is technically accurate. It's using the cache tag system correctly and is responding to a cache rebuild, which invalidates caches for all pages (in Page Cache, Dynamic Page Cache, or Tome), and that's why this behavior is happening. The information that Build Hooks collects is already available in Tome with cache tags (when an entity is updated its cache tags are cleared, when one is created it will show up as a new uncached path). "Fixing" or changing Tome's caching would require the module to ignore some core caching behavior, which I may have to end up doing. I'm re-assigning this to think about this and maybe work on it, but wanted to give some more information because I know this issue is a big performance problem for users.

samuel.mortenson’s picture

Status: Active » Needs review
FileSize
4.86 KB

I've been working on a small sub-module for Tome Static, "tome_static_super_cache", which does the following:

  1. Changes core caching behavior so that during a full cache rebuild, common cache tags like config:system.menu.*, http_response, and local_task are not invalidated.
  2. Changes Tome Static's caching behavior so that the cache can never be fully deleted.
  3. Adds a new command, "tome:super-cache-rebuild", which can be used to actually clear all the cache unconditionally.

This is working pretty well, with the Umami demo a "drush tome:static" run after a cache rebuild only re-generates 8 paths instead of the normal 64. The code is a little funky, but I'm OK putting this in a sub-module and documenting it on the website. @Heisen-blue what do you think?

samuel.mortenson’s picture

+++ b/modules/tome_static/modules/tome_static_super_cache/src/TomeStaticSuperCacheTagsInvalidator.php
@@ -0,0 +1,35 @@
+    $is_rebuild = FALSE;
+    foreach (debug_backtrace(0) as $info) {
+      if (isset($info['function']) && $info['function'] === 'drupal_flush_all_caches') {
+        $is_rebuild = TRUE;
+        break;
+      }
+    }

I might set another global on rebuild (in a hook_cache_flush implementation) to avoid performance issues here.

samuel.mortenson’s picture

FileSize
9.72 KB
7.34 KB

I've been having some fun with this and have added an experimental Views cache plugin I started working on awhile ago to work around core list cache tags. This sub-module will probably be a test bed for a lot of bleeding edge cache features like this. Everything still needs tests.

samuel.mortenson’s picture

FileSize
24.15 KB
20.29 KB

Here are some tests, and documentation to explain the new behaviors.

samuel.mortenson’s picture

I want to commit this sometime next week so it can be in the next release (beta4) so any review is welcome!

  • samuel.mortenson authored 4ff1743 on 8.x-1.x
    Issue #3023453 by samuel.mortenson, rozh: Need to re-export the whole...
samuel.mortenson’s picture

Status: Needs review » Fixed

Committed! Using the new sub-module, Tome Static Super Cache, should help a lot for people that can get away with it. Please open new issues if you find bugs with the new functionality.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.