Problem/Motivation
Follow up to #2203411: Convert drupal_get_library() into a service, currently library info is cached per extension. So, for example, a vanilla D8 install, front page, there are ~ 10 calls to different library extension cache items.
Proposed resolution
Consolidate the per extension caches into one item, which is loaded by the Library service. We can still use getLibraryByExtension, hence no api changes, the class just gets its cached data once.
Remaining tasks
Wait for #2203411: Convert drupal_get_library() into a service, Decide best approach for getting all extension library info, implement above, tests.
User interface changes
None
API changes
None
Comments
Comment #1
damiankloip commentedComment #2
damiankloip commentedComment #3
catchSince this is by extension, we'll need to have the active theme in the cache key, none of these should be requested before the theme is set so should be OK.
Also assuming this is for all discovered libraries. Some might not be used, or only on rarely visited admin pages etc. so might be worth CacheCollector here?
Comment #4
dawehnerYeah we should come up with a scenario and have a look at the needed amount of memory for it.
Comment #5
sunNot only the active, but all enabled themes.
The memory consumption should be small. Even smaller if we'd find a way to omit the default/empty properties (e.g., by caching the original/unprocessed data instead of the processed data).
Comment #6
catchPostponing this on #2228291: Move all plugin and other discovery data into a cache_discovery bin too. With that and the PhpStorage cache backend we might have come up with a way to avoid the i/o memory trade-off.
Comment #7
damiankloip commentedComment #8
damiankloip commentedWhoops, sorry!
Comment #9
catchMoving this to duplicate. If the local cache stuff turns out to be a dead end, we can re-open this.
Comment #10
damiankloip commentedSeems still beneficial to do this if we can? This will not yield much more data than some plugin definitions? Sorry, changing back to needs review so it gets your attention again :)
Comment #11
catchYeah I don't mind doing this additionally to the other one. It's a lot less complex than config pre-loading.
Comment #12
damiankloip commentedHere is an initial patch - not finished. Just didn't want to get too far with converting/adding tests until we get some consensus on the approach. So this will fail.
I think the best way of doing this is to use the cache collector. We can then just rely on trying to get what's requested, like now, rather than trying to discover everything up front. I have moved basically everything out of the LibraryDiscovery class, and moved it to a LibraryDiscoveryCollector class. This extends CacheCollector and aggregates all of the library info extension data. From quick tests on the front page, this patch made 12 less get() queries to the cache backend.
I am not sure about what I have done in this patch as all the library discovery logic lives in the new collector class. Which is probably not a good idea.
Anyway, this is a start we can start picking/criticising etc...
Comment #14
damiankloip commentedOk, added a new LibraryDiscoveryParser service. This gets injected into the LibraryDiscoveryCollector instead now. This separates the logic between simple cache collector implementation and the parsing logic to get library data for an extension.
Will fix tests once I get some validation/happiness with the approach.
Comment #16
dawehnerShit really? In general I really wonder whether it is worth to split this up into 3 classes tbh. As long you change the tests properly this though should work fine.
Comment #17
damiankloip commentedConverted LibraryDiscoveryTest to LibraryDiscoveryParserTest, added a new test for the LibraryDiscoveryCollector.
Comment #18
damiankloip commented17: 2223143-17.patch queued for re-testing.
Comment #19
dawehnerat least the doc does not tell you anything wrong :P
Comment #20
damiankloip commentedGood point!
Comment #21
sunLet's order these two alphabetically.
"buildLibrariesByExtension" is very long.
$this->discoveryParser->getByExtension($name)?Is there actually a different way than by extension, or do we intend to add one in the future?
If not, just LibraryParser->get($extension)?
Typo in "moduel"
Hm. If we're going to split it all up this radically, then I wonder why there is still a separate/custom/duplicate property cache on the parser?
Shouldn't the parser just parse when told to do so?
Btw, we still need to inject some
$typeknowledge into all these classes...Two possible approaches:
A. Use the legacy
$type, $namepattern ofdrupal_get_path()& Co.B. Use
Extensionobjects. (they have->getType()built-in)Comment #22
damiankloip commentedThanks for the review.
1. Fixed
2. I thought you added this originally ?! :p I think the buildByExtension option is good?
3. Fixed
4. Fixed, this was just a precaution more than anything. As discussed, as the caching layer worries about this now. This is just extra memory usage.
5. That's a valid concern. It is pre existing though, and if we can I would like to punt that to a follow up. This patch is big enough already. I don't think this issue would make that more difficult. If we needed to we could change the parameters of the parser and collector classes. Either way we would need to change the LibraryDiscovery interface parameters to achieve this (which is why I would like to isolate that discussion after this).
If you are happy, I will create a follow up for that so we can start discussing there?
Comment #23
damiankloip commentedComment #24
sunThanks!
Yeah, let's definitely create a follow-up for the extension type issue - that magic type detection hack was definitely meant to be temporary only...
It would also be great to create a separate follow-up issue to (1) rename the service IDs into 'asset.library_discovery*' and optionally (2) move the classes into a
\Asset\Discoverysubnamespace and remove "Discovery" from the class names.Comment #25
damiankloip commentedCreated #2279551: Library discovery Parser tries to guess extension type assuming 'theme' for non-existing modules and #2279553: Move library discovery into Core\Asset\Discovery namespace and add 'asset.' prefix to service IDs
Comment #26
alexpottCommitted c5942ee and pushed to 8.x. Thanks!
Comment #29
penyaskitoThe follow-up that never happened: #3536163: Library discovery Parser should allow a version constant for modules and themes