Problem/Motivation
While working on enhancing the Rules caching #2189645 I came across following scenario:
system_cron() invokes $cache_tables = array_merge(module_invoke_all('flush_caches'), $core); and there's entity_flush_caches() which calls entity_defaults_rebuild(). This entity rebuild, goes through RulesPlugin::save() which calls rules_clear_cache().
And there it is, the whole rules cache is lost, no matter if the items where expired or not. On a site with a lot of rules this can lead to stampeding since the rebuild of the rules cache hasn't locking yet (#2190553: Add locking to cache rebuild to avoid stampeding)
Proposed resolution
Features had a similar problem and introduced a variable to disable rebuilding on cache flushes.
We could reuse this and hook into the features rebuilding process.
This by implementing hook_features_rebuild() and run the entity rebuilding when it is invoked.
In entity_flush_caches() we can do a check for the variable of features. If it's there and disabled rebuild on cache flush it means features is installed will takes care of the rebuilding. If the variable isn't set / rebuild on cache flush is enabled we act as usual and do a rebuild.
Remaining tasks
Reviews needed.
User interface changes
None.
API changes
Introduced EntityDefaultFeaturesController::rebuild().
| Comment | File | Size | Author |
|---|---|---|---|
| #21 | entity_defaults_rebuild-2241979-19.patch | 3.84 KB | das-peter |
Comments
Comment #1
fagoRe-using the features variable makes sense to me.
Problem is this is a features component callback only - so when no feature component is in rebuildable state, e.g. you have nothing exported or it's all overridden - it won't be fired.
We'd need a way to kick in independently - so we now we can rebuild when features rebuilds its stuff also. Unfortunately, features does not provide a general hook to react on feature-rebuilding yet, so I guess we'd have to add something like that.
Also, a features-rebuild drush command would make sense if rebuild on cache clear is disabled - but that's another story I guess.
Comment #2
das-peter commentedUpdated approach after stumbling over #1844566: Invoke pre and post hooks when ALL components have been enabled and adding another patch there.
What about this approach? That should make the hook invocation components independent, right?
Comment #3
fagoYeah, this approach looks good to me. We'd probably want to add something to the README about it, so people are aware of this option to speed up things :-)
Comment #4
das-peter commentedYay the blocking issue was fixed: #1844566: Invoke pre and post hooks when ALL components have been enabled
Comment #5
das-peter commentedAttempt on writing something that is understandable - I think I failed though :P
Comment #6
fagoThanks on the great work on this patch and the features patch. I was little worried about people running features 2.0 and missing the new hook, but then I figured features_rebuild_on_flush is quite undocumented - so probably not a lot of people have it activated. I'll make sure to document this in the commit message still though.
Thus, I took your patch and committed it. README text was already good thanks, I tweaked it a little bit more though.
Comment #7
fagouhm, just realized it's not hidden at all - so this could be a problem then. Therefore I did not commit it for now. Attached is an updated patch with my README tweaks.
Let's improve it so it will continue working with features 2.0 and features_rebuild_on_flush set to TRUE also. Maybe best, let's just add an additional flag like entity_rebuild_on_flush = TRUE|FALSE and leave it to the people whether they use features for triggering rebuilds?
Comment #8
das-peter commentedI think the patch was lost :)
Comment #9
fagoops :)
Comment #10
das-peter commentedWell, if we're introducing a new variable anyway - how about making it entity type specific too?
Global:
entity_rebuild_on_flush = TRUE|FALSESpecific:
entity_rebuild_on_flush_[entity-type] = TRUE|FALSEComment #11
hefox commentedComment #12
fagoWhy would you disable it per entity-type? Is there a use-case you have in mind?
Comment #13
das-peter commentedHope I got it right.
Comment #14
socialnicheguru commentedWill this interfere with features updates or rebuilds using drush?
drush fr feature-name
drush fu feature-name
Comment #15
das-peter commented@SocialNicheGuru No
Comment #16
fagoThat line should be removed imho, the comment is enough.
Rebuild all defaults
Comment needs an update - its our own setting.
Unrelated.
Comment #17
das-peter commentedComment #18
a.milkovskyWhy do we need this code if all the logic is done in entity_features_post_restore()?
Comment #19
a.milkovsky#17 looks good for me. Cache clearing is way faster now.
Removed EntityDefaultFeaturesController::rebuild() as all the logic is done in entity_features_post_restore().
Comment #21
das-peter commented@a.milkovsky: Thanks for the update!
Re-uploading patch from #19 without the diff to get it green again.
Please use for interdiffs the file extension
.txtor add the suffixdo-not-testto avoid unrelated testbot execution / fails.Comment #22
fagoThanks, this looks ready to me and can go into the next release. Let's put it on the table for others too first.
Comment #23
fagoThanks, committed.
Comment #24
fagoComment #26
hefox commentedSo, now on every feature rebuild/revert, alll the entity defaults are rebuilt? That might cause a bit of a slow down for reverts -- I believe drush batches them so does a few at a time.
Comment #27
das-peter commentedIndeed, we added code that invokes a general rebuild - which also covers defaults not handled by features. If there are many default entities outside of features you will notice a slow down I guess.
Comment #28
hefox commentedIf you got 20 features, and only one has entity exportables, it'll do a rebuild each time any of those features are rebuilt, correct?
For example, during batched module enable during an install of a distro
Comment #29
das-peter commentedAll I can confirm is that it will do a full defaults rebuild whenever
hook_features_post_restore()is invoked. "Full" means all exportable entity types wherever they are provided (features or not).Comment #30
hefox commentedThat means whenever _features_restore with op == rebuild
That will slow down install of distros in that situation (not sure which distros that is/ what components are provided by entities). Probably would be better to test if the features being reverted provides an exportable provided by entities
Comment #31
a.milkovskyPreviously "Full" rebuild was executed on every cache clear. With this patch performance of cache clear is really increased. But @hefox is right, feature revert performance is definitely decreased.
Comment #32
fagoMaybe, is there a way we can improve that during batches and skip it if we did it already during a batch? Sounds like we should create a follow-up issue.
Comment #34
osopolarWondering, why is entity_rebuild_on_flush not set to FALSE by default? Are there any negative side-effects setting entity_rebuild_on_flush to FALSE?
Comment #35
osopolarComment #37
kenorb commentedComment #38
kenorb commented