Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Test scenario:
- Ensure #2430219: Implement a key value store to optimise config entity lookups is applying otherwise creating the configuration entities will take ages
- Install minimal
- Enable contact module
- Create 10 field storages on the contact message content entity
- Create 50 contact message bundles and add 10 field instances to each
- Delete all the field storages.
Doing the deletes...
Deleted field storage contact_message.field_map_storage_0 40.41s
Deleted field storage contact_message.field_map_storage_1 36.68s
Deleted field storage contact_message.field_map_storage_2 34.33s
Deleted field storage contact_message.field_map_storage_3 31.12s
Deleted field storage contact_message.field_map_storage_4 26.82s
Deleted field storage contact_message.field_map_storage_5 24.08s
Deleted field storage contact_message.field_map_storage_6 20.48s
Deleted field storage contact_message.field_map_storage_7 17.34s
Deleted field storage contact_message.field_map_storage_8 14.44s
Deleted field storage contact_message.field_map_storage_9 11.38s
Ouch.
This looks like it is due to #2473983: [meta] Evaluate Entity Field API Scalability.
My scripts to reproduce this are in attached.
| Comment | File | Size | Author |
|---|---|---|---|
| #5 | 3-5-interdiff.txt | 486 bytes | alexpott |
| #5 | 2482231.5.patch | 1.78 KB | alexpott |
| #3 | 2482231.3.patch | 2.46 KB | alexpott |
| alex_create.php.txt | 1.04 KB | alexpott | |
| alex_delete.php.txt | 465 bytes | alexpott | |
Comments
Comment #1
alexpottxhprof profiling the first field storage delete shows:
But the good news is this looks like it is Drupal\Core\Entity\EntityManager::getFieldMap since that takes 90% of the time! So #2473983: [meta] Evaluate Entity Field API Scalability looks the ticket to fixing this. Very happy it is not dependencies. Postponing to check again after that issue is resolved.
Comment #2
Wim LeersThe meta got more child issues. Notably #2482295: Rebuilding field map with many bundles/fields is very slow, which I think is the one that this was actually postponed on?
Reopening.
Comment #3
alexpottWith the patch attached...
Without the patch attached...
So significant improvements and things become more predictable and quicker.
Comment #4
BerdirNice.
The clearCachedDefinitions() removal overlaps a bit with #2487287: Optimize/clean up cache clears when saving/deleting FieldConfig entities but that's not ready yet, so we can always reroll it. I'm not sure I see how it can be just removed, though?
Comment #5
alexpottLet's handle the entity manager cache clearing part in #2487287: Optimize/clean up cache clears when saving/deleting FieldConfig entities. The reason I just removed the cache clear was because
The above line hinted that the only reason we were doing this was to refresh the results of getBundles - however this was made unnecessary by #2482295: Rebuilding field map with many bundles/fields is very slow
With the patch attached:
Without patch
So as we can see still a very significant speed up and peak memory is largely the same (well actually a little bit better - so the whole static cache worry in the original comment has proved to be unfounded).
Comment #6
alexpottThe numbers in #5 are with minimal - here are the numbers with standard (more config entities exist)
With patch
Without patch
Comment #7
BerdirThis seems like a good improvement. We'll continue testing and improving this in #2487287: Optimize/clean up cache clears when saving/deleting FieldConfig entities.
Comment #9
webchickCommitted and pushed to 8.0.x. Thanks!