Problem/Motivation
In #3565258: Support library-specific aggregates we added a 'libraries' parameter for asset generation, that sends the exact list of libraries required to generate an aggregate instead of include/exclude to generate it from the full context of the page. This means that if the same aggregate appears on different pages with different other aggregates, its URL is exactly the same including the query string, resulting in a better browser/CDN cache hit rate.
This is used by default for JavaScript, with a fallback to include/exclude when a library is split between different aggregates (because other files are in the middle).
Now that #1945262: Introduce "before" and "after" for conditional ordering in library definitions has landed we should also make it the default for CSS aggregates.
Once we remove per-file weight support and explicitly disallow adding files in hook_js_alter() we can also remove include/exclude support entirely, but that's a few steps away.
Steps to reproduce
Proposed resolution
Remaining tasks
User interface changes
Introduced terminology
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|
Issue fork drupal-3615654
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
catchComment #4
catchComment #5
nicxvan commentedI don't understand why the counts go up but the size goes down?
Comment #6
catchThe test visits three node add pages for different bundles which each result in a different combination of libraries being attached.
The count goes up because the total number of CSS or js files requested across those pages increased.
The size goes down because the total kb transferred is reduced, because there's less duplication of content between the files.
The number of files increase is likely because the library-specific aggregates are splitting what was previously one aggregate.
Say an aggregate contains libraries a,b,c,d,e
Even if that aggregate always contains the same five libraries on all three pages, the include query parameter can be different depending on other libraries that appear on the page. Because to reproduce the correct file, the include/exclude query string approach needs the full context of all libraries on the page as well as the delta of the file.
Eg. Page 2 could include library f and page 3 could include g and h and even though they don't affect the contents of our abcde aggregate, they theoretically might if one ends up in the middle etc.
So that one aggregate gets downloaded three times, counts as three files and three times the size.
When we add aggregate target to library c, what was one file becomes three.
a, b (include/exclude)
c (aggregate target/libraries )
d, e (include/exclude)
C is now exactly the same across all three pages so there is no duplication of that file. Because the libraries parameter is all that's needed to produce the aggregate correctly, meaning that other libraries on the page have no effect.
The a,b and d,e aggregates still get duplicated (even if only due to query string). But each of these now appear three times with different query strings.
So instead of 1 x 3 it's more like 2 x 3 + 1.
Once we fully remove individual file weight support we'll be able to exclusively use the libraries query string to generate aggregates. We might be able to do that already with some kind of detection fallback. That will remove the 'same file, different query string' problem.
However there's two further issues:
If page 1 generates an aggregate with libraries a,b and page 2 a,b,c, that's a genuinely different file so duplication still happens. Aggregate target helps here by isolating libraries from other ones.
The other problem is CSS groups (base, theme, component) split libraries up even without custom weights and the libraries approach can currently only handle one group per file rather than multiple groups. There are a couple of open issues to try to resolve that but it's either going to be very tricky logic or require a different way of organising CSS (like CSS layers).
Comment #7
catchWriting all that out, now that we've done #1945262: Introduce "before" and "after" for conditional ordering in library definitions it's worth trying to see if we can use the 'libraries' approach for CSS by default, same as we do for js - previously it just made things worse but that might not longer be the case.
Comment #8
nicxvan commentedThanks for writing that up, it seems that it brings us back to the question more connections with smaller size or fewer connections with larger size.
It sounds like this is better on average, I would be interested in seeing that libraries approach, any idea of the scope?
Comment #9
catchPushed a commit for the CSS libraries change. We might need to split this issue up a bit for scope but in terms of what the actual eventual behaviour looks like especially looking at performance tests, it's much easier to combine to start with.
The actual code change is pretty small, I had it like that at one point in #3565258: Support library-specific aggregates but the number of libraries with per-file weights in core really messed things up. Now those are gone things look a lot more reasonable. It's how js works already (to the extent that js doesn't have CSS categories). We still have a fallback to include/exclude if e.g. one file in a library with lots of files is excluded from aggregation since that'll split the files across aggregates again (or contrib/custom code that can still specify per-file weights). And we only support one category per aggregate because multiple categories gets very, very complicated or maybe impossible.
Yes although with http/2 it's a bit more flexible than it was ten years ago. The working number of concurrent streams per connection is 100 per https://dev.to/sibiraj/understanding-http2-parallel-requests-streams-vs-... - had some notes about this in #3565258-14: Support library-specific aggregates.
When http/2 was in development people were saying that aggregation was going to become pointless, and that's obviously false because without aggregations we'd go well over 100 files downloads between js, css, images, fonts. But it does mean that if we go from 5-10 css/js files to 10-20 cs/js files we're still probably leaving plenty of headroom for fonts/images and slower connections.
Comment #10
catchComment #11
catchComment #12
catchSplit the navigation change out to #3615642: Add css_target to navigation libraries.
Comment #13
catchSplit the ckeditor change out to #3622766: Use aggregate_target for ckeditor libraries and rebased.
The number of files actually goes down in core/profiles/demo_umami/tests/src/FunctionalJavascript/MultipleRequestsPerformanceTestAssertions/umamiFrontAndRecipePages.yml - this is will be 'libraries' aggregates appearing on multiple pages. We should eventually have more of that once we've gotten rid of custom weights completely and refactored more libraries. See also #3619507: Remove weights from system.libraries.yml.
In other cases the number of stylesheets goes up a bit, but the bytes goes down (in some cases by about 1/3rd) or stays flat.
Comment #14
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #15
catchRebased.
Comment #16
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.