Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.What I've done:
1) Load a JSON file containing data for 363 config entities (1 address_format, 362 subdivision config entities).
2) Start XHProf
3) Initialize each config entity, setSyncing(TRUE), save()
4) Stop XHProf
Attaching the xhprof. It clearly shows that significant time is spent invalidating cache tags (invalidateTagsOnSave()), and in ConfigEntityBase::preSave().
1) Would be great to find a way to invalidate cache tags only once per import.
This would require wrapping invalidateTagsOnSave() with a isSyncing() check, and having a way to invalidate the tags after the sync (a postSync() method?)
2) ConfigEntityBase::preSave() has this:
// Ensure this entity's UUID does not exist with a different ID, regardless
// of whether it's new or updated.
$matching_entities = $storage->getQuery()
->condition('uuid', $this->uuid())
->execute();
$matched_entity = reset($matching_entities);
if (!empty($matched_entity) && ($matched_entity != $this->id()) && $matched_entity != $this->getOriginalId()) {
throw new ConfigDuplicateUUIDException(String::format('Attempt to save a configuration entity %id with UUID %uuid when this UUID is already used for %matched', array('%id' => $this->id(), '%uuid' => $this->uuid(), '%matched' => $matched_entity)));
}
This results in an entity query per save, taking up 25% of the total time. Ouch.
Alex Pott tells me that it should be safe to wrap at least that part with an isSyncing() check, but I haven't been able to confirm.
| Comment | File | Size | Author |
|---|---|---|---|
| 54d8e45a81393.d8.xhprof.zip | 23.6 KB | bojanz |
Comments
Comment #1
bojanz CreditAttribution: bojanz commentedComment #2
catchWe could optimize the merge query implementation on MySQL, that would halve the queries here.
Not sure moving things to the end will help unless we're repeatedly invalidating the same cache tag?
Comment #3
bojanz CreditAttribution: bojanz commentedHalf of those 720 invalidations are for the same tag, done by ConfigEntityBase::invalidateTagsOnSave():
The others are done by the config system for the individual saved entities.
Comment #4
larowlanEntity query with config entities used to be a full scan each time, perhaps a uuid indexer service would be an option
Comment #5
Berdir85% of the invalidateTags() problem is ChainedFast, again. #2422657: Skip fast chained cache backend during installer is related, but that only fixed it for the installer.
I kind of caused this, because what it id before was broken.
Trying to remember where I was writing this before. Right, comment #24 in #2289201: [Meta] Make drupal install and run within reasonable php memory limits so we can reset the memory requirements to lower levels. We need an issue for that. markAsOutdated() needs to be optimized so that it is not called multiple times, except something happens where we need it again. Unfortunately, I'm not quite sure what that something would be, and thinking about it, not sure if that really helps. Because we are actually writing and reading from the chained fast backend as well, when writing/reading.
But maybe a single mark as outdated for cache tags per request would be enough? Or maybe, just like we changed it for the installer, we shouldn't use the fast chained backend during during a config sync, but only do a single explicit invalidation (The problem once again is that we don't know all cache bins in the factory, so we don't know what to invalidate until it is actually called).
Comment #6
Wim LeersRelated: #2241377: [meta] Profile/rationalise cache tags
Comment #7
bojanz CreditAttribution: bojanz commentedComment #20
BerdirI'm pretty sure this is outdated, cache tag invalidation will not re-invalidate the same cache tag more than once in a row and #3334489: ChainedFastBackend invalidates all items when cache tags are invalidated just landed, plus the install stuff mentioned above many years ago.
On 2, I'm not 100% sure it's safe to skip the check, you could absolutely copy & paste a config entity manually, forget to change the UUID and try to import it. I think back then we didn't have config entity query indexes, so this should be much faster now.