Part of the automated tests for the Thunder distribution is performance regression tests. We test various editorial tasks by executing them as Nightwatch tests and sending time markers to an Elastic APM stack at various stages of the task.
We then run those tests multiple times for different branches of the distribution and compare the results. Two branches we always test are a branch with the current Drupal version (Drupal 8.8.1 as of now) and another one with the latest version of the previous Drupal minor version (which is currently Drupal 8.7.11).
The test that measures the time between clicking on the "Save" button on a node edit form and the fully-loaded target page has seen a noticeable regression between Drupal 8.7 and 8.8.
This is a visualization of multiple test runs
The yellow line represents Drupal 8.7, the green one 8.8.
The average time suggests, that Drupal 8.8 takes about 15% longer to execute the node save.
It has to be said, that those tests are not run with Vanilla Drupal, we will provide easier reproducible tests later. But we wanted to get this getting started now.
Some things to consider
| Comment | File | Size | Author |
|---|---|---|---|
| #10 | load-content-view.png | 83.14 KB | daniel.bosen |
| #6 | node-load.png | 64.11 KB | daniel.bosen |
| grafana.png | 95.81 KB | daniel.bosen |
Comments
Comment #2
daniel.bosenComment #3
daniel.bosenComment #4
wim leersAre you also observing performance differences when just viewing e.g. article pages when the cache is cold? (Page Cache, Dynamic Page Cache, Render Cache, any or all of these.)
Because the report here is not about the saving itself per se, it's possible that this is a regression solely on the rendering side.
Comment #5
wim leersAlso: what is being measured exactly? It sounds like it's the
window.onloadevent. It'd be helpful to know if the server response time has changed (implying it's a server side performance regression), or if it's entirely a front end matter (client side performance regression).Comment #6
daniel.bosenWe do not have data for the node view, but for the form load of the edit form. There is no big difference in 8.8 to 8.7.
Comment #7
daniel.bosenRegarding #5 we are still improving the test, and try to get better results. Currently we measure the time from "Click save button" to "content list is visible on admin/content page".
So currently it is uncertain, where the time actually lost. Working on it :-)
Comment #8
wim leersAwesome :)
And to be clear: I'm super grateful that you have this infrastructure set up and are taking the time to report back! 🥳
Comment #9
hchonovAmazing. It would be great if we have some setup like this for core as well :). I think this most probably belongs in the entity system.
@daniel.bosen, could you please name the core modules you are using? Do you use any module relying on forward revisions like content_moderation or workspaces? Are they enabled for the node entity type in your tests?
Comment #10
daniel.bosenFurther investigations seem to indicate, that the real culprit is loading the content list after the save.
This is a graph of the durations for server-side loading of the content list, so no save post request and no rendering of the content list. The absolute time difference is very similar to the time difference of the whole process.
The times are taken from the browser measurement of requests, not within PHP.
Comment #11
wim leers@hchonov For that, see #2308683: [meta] Create a 'Profiling' install profile, for testbot-powered simple profiling and easier local profiling and #2497185: [no patch] Create standardized core profiling scenarios and start tracking metrics for them 😄
Comment #13
daniel.bosen@hchonov we disabled content moderation for this and do not use workspaces. We also disabled the custom view we usually use for admin/content.
Comment #14
daniel.bosensorry for deleting the related...
Comment #15
AndyThornton commentedSorry, no fancy graphs here (nice work!), but, yeah, we think we've seen a slow down too. We have some smoke tests that generate product pages and add layout builder blocks and we've had to tweak the 'sleeps' to be a bit longer. Does seem to be relatively margin but appreciable.
Comment #16
hchonovI don't know how to proceed here. The best would be to narrow it down to a specific commit, but I don't know if this could be automated.
@daniel.bosen, maybe you could randomly pick some commits and check when the performance regression started occurring.
Comment #17
daniel.bosenWe were occupied with other things in the last few weeks, but I will continue investigating this now. First I try to git bisect the problem, as @hchonov suggested.
Comment #18
alexpottHere's the first sub issue to address performance regressions in 8.8.x - see #3131585: Performance regression caused by using the last installed entity definitions
Comment #20
andypostProper branch
Comment #26
quietone commentedHas this been solved by the commit of the single child issue, #3131585: Performance regression caused by using the last installed entity definitions?
If we don't receive additional information to help with the issue, it may be closed after three months.
Thanks!
Comment #27
smustgrave commentedSince there has not been a follow up going to close this one out. If still an issue in D11 please reopen.