Problem/Motivation
I frequently see scheduler_form_alter() show up in blackfire and while some reports around it are strange, there are definitely some very unusual things here.
getPlugins() uses a persistant cache for instantiated plugins. I'm fairly certain that's more expensive than just instantiating them, especially because there is only a persistent cache.
However, getEntityFormIds() is quite expensive, because it calls into each plugin, which loops over all bundles and loads them and so on.
And it's doing that not just once, but multiple times due to the multiple form ids checks.
Steps to reproduce
Proposed resolution
Change the persistent cache to a property in getPlugins() so it's done only once per request.
Instead, add caches to simple outputs, such as getEntityFormIds() and also those route methods, as they are apparently also called on every request, such as due to \Drupal\scheduler\Theme\SchedulerThemeNegotiator::applies. Since this caches have no variations and are small, they can go into the fast chained bootstrap or discovery bin. They will however need cache tags, for example for the bundle-based things.
Remaining tasks
User interface changes
API changes
Data model changes
Issue fork scheduler-3545432
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
berdirAs an additional improvement, I'd add a check to scheduler_form_alter() to see if this is even an entity form ($form_state->getFormObject() instanceof EntityFormInterface, at least for the methods that expect it.
Comment #3
berdirI see this was already improved recently in #3533739: Implement static caching around SchedulerManager::getPlugins() but I'd go further and remove the persistent cache completely.
Comment #5
berdirCreated a merge request with some suggestions.
With the change in the other issue, it's better, but it still shows up in blackfire. Converting to only a property, getPlugins() no longer shows up at all, and by converting getEntityFormIds() to fast chained cache, scheduler_form_alter() is now no longer visible in blackfire as well.
Additionally, I changed the getTypes() call to using the bundle info service, that's faster (loading all config entities just to get their ids is very expensive), and it would also work for entity types that don't use bundle config entities. That could also be skipped for this issue, didn't do DI and so on.
Comment #6
berdirWill need to check the failing tests.
Comment #7
jonathan1055 commentedThank you Berdir, this looks good.
Comment #8
jonathan1055 commentedI pushed a changed to filter for just the failing tests, to focus on those. When passing we can go back to the full set.
https://git.drupalcode.org/project/scheduler/-/jobs/6484435
However this means running
_PHPUNIT_CONCURRENT: 0i.e. using the plain phpunti binary, without concurrency, which is slower. I think there is a better way to filter on just a few test classes and still have concurrency using run-tests.sh.Comment #9
berdirThis should help with the tests, I had misremembered the cache tag that's used for the bundle info. Also added tests for he other one.
The use of the entity type bundle info service should enable considerable simplification in SchedulerPluginBase. It will always return a default bundle, so the whole $isBundle stuff can go. getTypes() is unused now.
Not new, but it makes no sense that $entity_type is passed to entityFormIdsByType() because it's already available as the plugin is for a specific entity type(update, I see it's for the bundle entity type that's passed, still pretty confusing and at least the is bundle part can be dropped). The protected properties in \Drupal\scheduler\SchedulerPluginBase::entityTypeFormIds and \Drupal\scheduler\SchedulerPluginBase::entityFormIds() are likely pointless as well now, there are no direct calls to these methods. But all that could be a follow-up and some of it is tricky with BC if you care about that for child plugin classes.Comment #10
jonathan1055 commentedThanks for the updates and its great that the tests are passing. Nice when it's something simple to fix.
If you wanted to make these chages, please go ahead now. Or we could commit MR254 then keep this issue open and make those changes in a second MR here too. You know what you are doing, so it make sense for you to do it whilst you have it in your head.
Comment #11
jonathan1055 commentedActually, I've changed my view on #12 and maybe it would be better to do the simplification and removal of redundant code in a separate MR in a follow-up issue, and keep MR254 just for the caching improvements. Then it will be easier to see what is being changed/removed later, after this one is committed.
Comment #12
berdirI agree. Part of the cleanup would be possible without BC breaks, but I don't know your exact approach on things like removing properties, arguments and methods in a plugin base class. so it's probably easier to do that refactoring, which will not have a performance impact, in a separate issue.
I was able to work on this partially in scope of a performance assessment on a client site, any further work would need to happen outside of that I can't say if and when I'll be able to do that.
It would also be very nice to have the already done improvement and this in a release, it's currently hard to apply those changes as patches due to larger changes.
Comment #13
jonathan1055 commentedFor BC, the biggest dependent project is Scheduler Content Moderation Integration so we'd need to test thoroughly with that. I want to run a test on SCMI anyway using this MR254 before merging, just to make sure.
The test coverage we have in Scheduler is pretty thorough (I have made sure of that over the years) so I'm fairly confident that your changes here are good to go in. Yes a new release is worth it, for these improvements.
Comment #14
jonathan1055 commentedI have tested this with SCMI on the 3502119-pipeline-test-3.x branch. The first commit applied the patch from this MR but the phpunit job failed becuase the patch does not apply on the latest Scheduler release. So the next commit used Scheduler 2.x-dev and that worked. The phpunit jobs ran OK
https://git.drupalcode.org/issue/scheduler_content_moderation_integratio...
So I think this is ready to commit, I can't think of any further testing to do.
Comment #16
jonathan1055 commentedThank you @berdir for this excellent enhancement. I will make a new release soon.
Comment #18
jonathan1055 commentedI have released Scheduler 2.2.2
Thanks again Berdir for providing this significant performance improvment. If you have any notes or ideas about the tidy-up please add them to the follow-up issue #3553247: Remove redundant code after caching improvements