Problem/Motivation
Doing some profiling...
CommentManager::getAllFields() is called in comment_node_links_alter(), if you display 100 nodes then it is called 100 times, which takes 3.5ms/1.3% of the page request time in my case. Considering that calling EntityManager::getFieldMap(), which is statically cached, only takes 260microseconds (7%), that's how fast it would be if we'd have a static cache directly in that method.
Proposed resolution
Build another statically cached map in entity manager keyed by field type.
Remaining tasks
review, profile, commit
User interface changes
no
API changes
Added EntityManagerInterface::getFieldMapByFieldType()
Removed CommentManagerInterface::getAllFields()
Original report by @Berdir
Doing some profiling...
CommentManager::getAllFields() is called in comment_node_links_alter(), if you display 100 nodes then it is called 100 times, which takes 3.5ms/1.3% of the page request time in my case. Considering that calling EntityManager::getFieldMap(), which is statically cached, only takes 260microseconds (7%), that's how fast it would be if we'd have a static cache directly in that method.
| Comment | File | Size | Author |
|---|---|---|---|
| #45 | comment-mgr-2269025-45-interdiff.txt | 911 bytes | berdir |
| #45 | comment-mgr-2269025-45.patch | 12.21 KB | berdir |
| #44 | comment-mgr-2269025.interdiff.42-44.txt | 811 bytes | penyaskito |
| #44 | comment-mgr-2269025-44.patch | 12.57 KB | penyaskito |
| #42 | comment-mgr-2269025-42.patch | 12.69 KB | penyaskito |
Comments
Comment #1
berdirComment #2
larowlanHere's a failing test to get you started, your challenge, should you choose to accept it - is to make this patch green.
Comment #3
andypostComment #5
grom358 commentedThe potential issue with this patch is if EntityManagerInterface::clearCachedFieldDefinitions() is called during a request, it doesn't result in cache clear for CommentManager::getAllFields().
Comment #6
larowlanLooks good to me
Comment #7
berdirYeah, almos too god, actually :)
This means that we can't separate between not being executed yet and an empty array.
Wondering if it's worth to optimize and differ between not-yet-executed and executed-but-no-fields-were-found, which means that we'd set the initial value to NULL and check with isset().
I suspect that is the reason this is not failing right now, because all tests just set up one field and we re-run this until we actually have a field and then don't need to worry about the static cache going stale?
Should we extend one of the web tests to delete the field and then check that th list was updated I suspect it would return stale information and we'd need to introduce a way to reset the fields.. that's always the downside of adding more caching :)
Comment #8
larowlanSo what if we added a reset method and then some timely hooks to fire to call the reset on update/delete/insert of field_config entities of type comment?
Comment #9
grom358 commentedOr on EntityManagerInterface have getFieldMapByType() method or getFieldMap($type) or something like that, and have EntityManager cache the filtered fieldMap for CommentManager and when fieldMap cache is cleared with clearCachedFieldDefinitions() is also clears the filtered fieldMap.
Comment #10
grom358 commentedPatch using EntityManager to handle the caching, so its refreshed when fieldMap is.
Comment #11
andypostMakes sense! Let's just move CM method to EM!
Suppose issue should change title and summary.
remains?
there's no sense to handle this method once EM does the same
this should test new entity manager method
Comment #12
mikeegouldingNew drupal contributor here, Mike Goulding, I'm at the Drupalcon Austin sprint. I noticed this needs to be re-rolled and I'm going to take care of that now.
Comment #13
mikeegouldingRerolled patch from comment #10.
Comment #15
moshe weitzman commentedThis has once again shown up as a performance problem in HEAD. If anyone is available to give this a final push, you will be kissed by the moon and stars.
Comment #16
penyaskitoWorking on it.
Comment #17
penyaskitoI started back from last green (#10). I don't fully understand what is asked in #11.
Comment #18
andypostLet's remove unused method now from comment manager
Comment #20
penyaskito$this->discovery does not exist.
Comment #26
penyaskitoPlease ignore #20.
$this->discovery is an attribute inherited in the parent class, and theorically is correctly initialized from constructor.
I cannot find what is wrong there.
Comment #27
andypost@penyaskito I think we need to extend existing EM unit test, no reason to test CM because EM gets new method
Comment #28
penyaskito@andypost But the failure comes from BreakpointMediaQueryTest :-S
Comment #29
berdirNope, the failing test is Drupal\comment\Tests\CommentManagerTest->testGetFields, cd core; ./vendor/bin/phpunit to see the backtrace (if you have xdebug enabled).
That test mocks EntityManager without calling the constructor, that's why it's not there. And yes, it doesn't make sense to test this method in CommentManagerTest.
Comment #30
andypostFixed CM test, still needs to extend
core/modules/comment/tests/src/CommentManagerTest.phpComment #31
moshe weitzman commentedIncreasing priority due to performance
Comment #32
andyposthave no time now, the issue needs extend entity manager test to cover new method
Comment #33
grom358 commentedYeah i'm been too busy with other things to come back to this. So all that is left is to add tests to EntityManagerTest ?
Comment #34
yched commented+1 on the approach, never really liked CM::getAllFields() :-).
Just a minor note that the phpdoc for the new EM property is a bit weird.
Comment #35
yched commentedAlso, there's probably some code in file.module that could leverage that new feature ?
Comment #36
penyaskitoIs this the way to go for the test? I'm not sure if I understood correctly.
Comment #37
andypost@penyaskito yes, it is!
Also needs to check #35
Comment #38
penyaskitoAdded test at #36 as proper patch. Small addition of some asserts.
Changed the phpdoc as @yched suggested in #34, not sure if this is more convincing.
Checked file.module for extending the usage of the new methods in there as @yched suggests in #35, but couldn't find any case.
Comment #39
berdirI don't think "Collect" here makes sense? It might do that internally, but that's an implementation detail that is not relevant for the API/Interface. So just, "Returns"?
I'm not sure about CommentManagerTest. It's not a unit test of CommentManager, it's mostly testing the static cache in EntityManager now, for which we now have a dedicated unit test. IMHO, we should either simplify that to mock getFieldMapByFieldType() and just verify that we call that method or remove it.
AFAIK, the only other call to getFieldMap() that we could update is is in Vocabulary::postSave()
Comment #40
penyaskitoComment #41
penyaskitoChanged phpdoc in EntityManagerInterface.
Changed CommentManagerTest for mocking getFieldMapByFieldType().
Reused getFieldMapByFieldType() in Vocabulary::postSave().
Comment #42
penyaskitoOops, fixed indentation.
Comment #43
berdirYou can probably simplify this to just getMock('...EntityManagerInterface') because we no longer have to call actual methods but just the one mocked method?
Comment #44
penyaskito@Berdir is right.
Comment #45
berdirUnit test is very long, removed a few distracting lines that are not necessary for this, was considering to mock getFieldMap() so that we can just the specific method, but that would be way too much to still be able to RTBC this.
I don't think this needs specific profiling, improvements should be pretty obvious.
Comment #46
catchNice find. Committed/pushed to 8.x, thanks!
Comment #48
penyaskitoKissed by the moon and the stars, thanks!
Comment #49
penyaskito