Problem/Motivation
Follow-up from #3334489: ChainedFastBackend invalidates all items when cache tags are invalidated.
Several core plugin managers use cache tags for invalidation - for example the element info cache depends on the theme, so an 'element_info' cache tag is added, and that way the cache can be invalidated for all themes using the tag.
This is also done in at least some places by language.
This adds some overhead to cache gets, because the cache tag needs to be fetched. However, we know that the cache tag is not cleared arbitrarily, it's only used in ::clearCachedDefinitions()
element_info_build
* theme_registry
* library_info
Steps to reproduce
Proposed resolution
Instead of using a cache tag, loop over the list of themes in ::clearCachedDefinitions(), clearing each cid individually.
Remaining tasks
Anything straightforward I think we could do in one issue. There might be some other discovery mechanisms (breakpoints etc.) where we want to open spin-off issues.
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #42 | Screenshot 2023-07-09 224236.png | 18.49 KB | berdir |
Issue fork drupal-3335768
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
berdirthere are some cache tag invalidations for library_info and theme_registry, for example \Drupal\system\EventSubscriber\ConfigCacheTag::onSave. That's why I was asking whether we considering that an API and therefore BC break.
Comment #3
andypostHow it will clear theme on its uninstall?
Comment #4
catchhmm I think that's actually wrong, and that it should be doing that via ::clearCachedDefinitions() on the plugin manager, if that's what it wants to clear. However we might want to open a spin-off issue for that to make that change first.
Comment #5
berdir> How it will clear theme on its uninstall?
It won't be requested anymore when it's uninstalled so that doesn't really matter. That said, module uninstall empties all cache bins and we could do the same for themes if we'd want to.
Comment #6
berdirSome work, we'll see how well it works. Didn't touch the language plugin managers yet. library_info usage is too dynamic, we can't replace that one.
Comment #7
andypostfix to see test result
Comment #8
andypost23 failed tests
Comment #9
berdirComment #10
berdirI missed the module theme build cache.
Comment #11
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #12
andypostFixed CS
Comment #14
berdirThanks for fixing my cs issues ;)
Down to two failures now, great, and that seems to be just a unit test that we'll need to update.
And the per-language plugin caches need to be updated too.
Comment #15
berdirFixed the unit test and updated some other plugin managers that use cache tags.
* help topics has an extension cache tag, but all plugins are invalidated when modules are installed or uninstalled, so that shouldn't be needed at all.
* contextual links, removed for now, but this also has group caches that do _not_ use the tag, so I think those are already broken. The thing is that I doubt that these are needed at all. it also doesn't do a reset of the data on the property. The only calculation is a single loop and checking them against the group, that should be quite fast? And I'm pretty sure we load all when a definition is then fetched for them.
* local actions
Kept it for local task manager, as that has per-route caches.
Comment #17
andypostHelp topics having twig extensions for routes/links and search module integration required hack for reindexing
Comment #18
berdirAll plugin types depend on which modules are enabled, that's why w clear them all through the plugin.cache_clearer service. However, we're not yet doing that for themes, which the cache tag supported. I added the cache clear call for that case instead, this doesn't cover uninstall yet, we probably need that too.
Comment #20
vladimirausComment #21
smustgrave commentedWonder if a simple change record could be created to announce the new function
clearCachedDefinitionsComment #22
berdir> clearCachedDefinitions
This is not a new function, it's just implementing an existing API of plugin managers.
But yes, this does need a change record, to announce the cache tags that are no longer available for invalidation and that instead those (existing) methods must be used. This should be very rare, there's a single case in core.
Somehow the previous changes from the patches didn't make it into the merge request, added that back now. Setting back to needs review to run the tests, this also needs reviews.
Comment #23
smustgrave commentedSeems to have a CI failure in the MR.
Ignored error pattern #cache tag might be unclear and does not contain
the cache key in it.# was not matched in reported errors.
-- --------------------------------------------------------------
Comment #24
berdirFixed that I think.
Comment #25
vladimiraus+1. Happy with the changes and tests.
Comment #26
berdirStill a bit of work left, and we need to create the change record.
Comment #27
berdirRerolled and updated constructors. Created https://www.drupal.org/node/3355227.
Comment #28
andypostComment #29
berdirFixed that in the gitlab editor. FWIW, I'm not convinced that cache tag invalidation is now in a better place. I think both module and theme install and uninstall should consistently clear caches, but that's not in scope of this issue.
Comment #30
andypostThere's still CS errors and it needs follow-up for #29
Comment #31
berdirMeh. ;)
Comment #32
andypostLeft few review comments, it looks mostly ready
Comment #33
berdirThank for the review, cleaned up ElementInfoManager and also added the trait for the removed property.
Comment #34
smustgrave commentedSeems there are 2 open threads on the MR still.
Think this should be updated for 10.2 now also?
Comment #35
berdirAs commented on the two open and related threads, I haven't seen mixed constructor declarations in core before, are you sure that is a thing we do? Setting to needs review to get more feedback.
Comment #36
andypostThere's in core https://git.drupalcode.org/project/drupal/-/blob/10.1.x/core/lib/Drupal/...
Moreover there's more
So I added commit with this change
Comment #37
andypost@Berdir thanks for review, fixed wrong condition
Comment #38
smustgrave commentedAll the threads seem to be resolved.
I was looking at the change record and think it would help to mention that
ConfigCacheTagnow requires\Drupal\Core\Theme\RegistryandElementInfoManagerno longer takes\Drupal\Core\Cache\CacheTagsInvalidatorInterfaceComment #40
berdirRebased the MR on 11.x. #3334489: ChainedFastBackend invalidates all items when cache tags are invalidated is in, so would be nice to land this as well to fully benefit from it.
#38: Personally, I find change records for trivial constructor changes for classes that are rarely if ever subclassed not useful. It doesn't belong in the change record that's about the actual change as it's related to that, and a separate one just for that is IMHO just noise and makes it harder to find relevant changes. The only afffected contrib module seems to be bootstrap for Registry, there are some subclasses of ElementInfoManager but they don't change the constructor.
But if that's blocking getting this to RTBC I'll create one.
Comment #41
catchAgreed we don't need change records for constructor changes. Not really here but will try to review this again soon.
Comment #42
berdirDid a little bit of profiling, pretty much as expected. Not a big change, but with render caching disabled on standard install on frontpage as admin, we're saving 3 calls to DatabaseCacheTagsChecksum::getTagInvalidationCounts(), each of them did a lookup for a unique and otherwise not used cache tag, so we save 3 queries. There are more plugin managers being changed, but they're not all being used in this cenario (link relations, help?, migration, ..).
Comment #43
smustgrave commentedPutting in committer queue.
Also removing CR updates tag per #41
Comment #45
catchUpdated the deprecation messages to 10.2 on commit. Committed/pushed to 10.2.x, thanks!
Comment #47
wim leersFYI this caused a regression in
migrate_pluswith no deprecation error: #3413533-5: Fix failing tests on HEAD.