Problem/Motivation
In SourcePluginBase, if a cache_key is not specified, one is generated by taking a sha256 hash of the source pluginID.
if (!isset($this->cacheKey)) {
$this->cacheKey = hash('sha256', $this->getPluginId());
}
This is incorrect, as the same source plugin may be used in multiple migrations with different configuration which affect the source count. For example, derived D7 node migrations will use the same d7_node source but with different bundle configurations to retrieve each content type, and each content type will have a different count. Due to #2598670: Source count caching does not work for SQL sources, the cache doesn't come into play for sql sources, which would cover most of core's use case, but if multiple migrations use say the contrib 'url' plugin and enable cache_counts without setting a cache_key, they will all interfere with each other.
Proposed resolution
Set the default cache key to a hash of the entire configuration array, and not just the plugin id.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #6 | 3092227-6.drupal.patch | 7.4 KB | mikelutz |
| #6 | 3092227-6.TEST_ONLY.drupal.patch | 5.49 KB | mikelutz |
Comments
Comment #2
mikelutzHere's a test.
Comment #3
mikelutzAnd a possible fix
Comment #4
mikelutzAnd you know patches with the right annotations, just for fun.
Comment #5
mikelutzAlright, I guess core_version_requirement: * doesn't work. ^8 || ^9 probably would but I don't want to leave it in there, so lets see if #3072702: Core extensions should not need to specify the new core_version_requirement in *.info.yml files so that 9.0.x can be installed gets committed soon.
Comment #6
mikelutzComment #8
heddnNo problems here with the implementation. Just some minor nits on on comments.
I'm a little confused with why we are statically calling SourcePluginBase::count() here. Perhaps an inline comment would clarify.
This doesn't really verify caching as best as it could. However, we do have more testing of that later. Maybe just a comment clarification of what is really being tested is in order.
Comment #11
wim leersAFAICT this is a duplicate of #2751829: Default source plugin cache key is insufficiently unique?
Comment #12
mikelutzI closed the other issue as a duplicate since this one has work done in it, and the other does not.
Comment #13
wim leersMarked #2751829: Default source plugin cache key is insufficiently unique as a duplicate of this.
EDIT: cross-posted, hah!
Comment #14
wim leersThis is also going to result in cache collisions, because now the plugin ID is ignored entirely.
It should be cached by both plugin ID and its configuration.
Now … you might say … that in practice, migration definitions have
plugin: <source PLUGIN ID>. But that's not actually necessary.Furthermore, having a cache ID that is literally just a hash is going to be a DX nightmware.
So I propose to keep your logic but prefix it with the source plugin ID 🤓 WDYT?
Comment #15
wim leersSee #3190815-28: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O). 🤞
Comment #16
mikelutzPrefixing the plugin_id or even using a serialized array with plugin_id and config_hash keys for readability would be fine with me.
Comment #17
wim leersCool!
Comment #18
quietone commentedThis is a blocker for #2598670: Source count caching does not work for SQL sources
Comment #20
alexpottI think this closed now that #3190815: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O) has landed. Changing to 9.1.x since that issue landed in that branch.