Problem/Motivation
CSS and JavaScript aggregation previously generated files inline during HTML page requests. If the file was requested without a PHP request to the page, and didn't exist, there was no way to create it and you'd get unstyled pages or broken JavaScript. This could happen just from a clear of js/css files on the server and then HTML cached in a CDN.
However it also had to periodically delete css and js files to avoid a potentially infinite collection of files.
To get around this only files older than 30 days are deleted when there's a cache clear.
However, following #1014086: Stampedes and cold cache performance issues with css/js aggregation, as long as a file is valid for libraries available on the site, we can recreate it independent of the 'parent' HTML request, and don't have to worry about the race condition.
Additionally, we store all the created files in state, this is necessary because the asset filename is based on the content of the files themselves, and that would be too expensive to recalculate. The asset filename with the new system is cheaper to recreate, so we don't really need this extra level of i/o caching there at all.
Steps to reproduce
Proposed resolution
Remove the css.cache_files and js.cache_files state entries.
Remove the system.performance stale file threshold.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #45 | 3301573-45.patch | 1.92 KB | catch |
| #45 | interdiff-10.1.txt | 1.01 KB | catch |
| #35 | 3301573-10.1.x.patch | 1.96 KB | catch |
| #33 | 3301573-33.patch | 13.57 KB | catch |
| #32 | 3301573-27.patch | 13.54 KB | catch |
Comments
Comment #2
catchComment #3
catchComment #4
catchComment #5
ravi.shankar commentedAdded reroll of patch #3 on Drupal 10.1.x.
Comment #6
catchComment #7
catchNote that the most recent patch on #2258313: Add license information to aggregated assets currently relies on this entry, but I think it's doing so unnecessarily. Even if similar information is needed, then it would make more sense for that issue to track it explicitly.
Comment #8
catchComment #9
anchal_gupta commentedPlease Ignore this patch. By mistake blank patch has been uploaded. I will again upload in the same thread
Comment #10
anchal_gupta commentedAdded reroll of patch #8 on Drupal 10.1.x.
Comment #11
catchtrailing space.
Comment #13
catchComment #14
catchComment #16
nod_Looks good, update hook as well. Looks like it wasn't tested so better to not have the code around :)
Comment #17
alexpottNeeds a bit of re-roll due to changes to system.post_update.php. I'd handle that on commit but I've got a question. Given we've only deprecated \Drupal\Core\Asset\CssCollectionOptimizer and \Drupal\Core\Asset\JsCollectionOptimizer and they both still use these state entries shouldn't we wait until Drupal 11 for this?
Comment #18
catch@alexpott hmm, that might be reason to delay the update to Drupal 11 (although those classes would just add the state entry back if they were being used, and then it would be cruft), however all the changes apart from the update would need to land in Drupal 10.1 so we can remove the new deprecations in Drupal 11.
Comment #19
ravi.shankar commentedAdded reroll of patch #14 on Drupal 10.1.x, and removed needs reroll tag.
Comment #21
catchRandom failure.
Comment #22
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #23
ameymudras commentedRerolled the patch
Comment #24
smustgrave commentedRunning PHPStan on *all* files.
------ ---------------------------------------------------------------------------
Line core/tests/Drupal/Tests/Core/Asset/CssCollectionOptimizerLazyUnitTest.php
------ ---------------------------------------------------------------------------
55 Class Drupal\Core\Asset\CssCollectionOptimizerLazy constructor
invoked with 11 parameters, 10 required.
115 Class Drupal\Core\Asset\CssCollectionOptimizerLazy constructor
invoked with 11 parameters, 10 required.
Comment #25
mrinalini9 commentedUpdated patch #23 by addressing #24, please review it.
Thanks!
Comment #26
catchThe interdiff looks good.
The last thing here is to deal with #17 and #18, I think we should probably remove the post update hook from this patch, and open a follow-up against Drupal 11 to add it there - can just have that part of the patch attached. Marking needs work for that.
Comment #32
catchMarked #3370828: Ensure that edge caches are busted on deployments for css/js aggregates as duplicate.
Re-rolled. Thinking about #17/#18 more I think if someone somewhere really is using the old classes, they should be responsible for cleaning up the state entries too, so leaving the update in. Crediting people from the other issue.
A workaround if you wanted this in 10.0.x would be to set stale_file_threshold to 0.
Bumping to major since this is a DX issue for local development, shame it stalled.
Comment #33
catchFixing cs issues.
We could also backport the ::deleteAll() changes to 10.0 without the deprecations. That would improve things for local development and deployments.
Comment #34
lauriiiTo me #32 sounds reasonable. Even if @alexpott disagrees, it seems fine to run the post update now and we could have another post update later to clean it up again if he feels strongly about this. I don't think there are any usages in contrib.
Went through contrib uses of this class and didn't see any calls for this. Seems fine to make this change and deprecate the method.
+1 for backporting the changes to
\Drupal\Core\Asset\CssCollectionOptimizerLazy::deleteAllto 10.1.x.Comment #35
catchHere's the 10.1 backport.
Comment #36
andypostCould be fixed on commit - s/11.1/11.0
Comment #37
mherchelWill this fix the issue in 10.1 where if a user clears cache, it doesn't refresh the aggregate CSS/JS that's pulled down?
Comment #38
nsciacca@mherchel - I just applied this to my 10.1 and other than a warning "] rmdir(/files/css): Directory not empty FileSystem.php:267" it did appear to fix the aggregate CSS/JS on my Pantheon deploy.
Comment #39
catch@mherchel yes it should.
Comment #40
mherchelJust tested this out, and it doesn't appear to be working.
Not sure if this blocks this issue, or if this should be reported in a followup.
Testing steps
/sites/default/files/css/css_hegpbZvWLCegz1eocO6sYJ0X_AGboa15OU4bOdIZPaI.css?delta=0&language=en&theme=olivero&include=eJx9UFuSgyAQvBDqGfYkqQF6lV1kqBmI6-0XEzWVVCo_MPQDmnYsGLzUTLHPwqNA1bgN5ORwnyKT_9nRXephud4FqeCvVIoH9UC6GNKvGl21YB4sKZ7uIFfCFTeR4dhG4SHzAoHv7NrZyO5BfAPeXAMWHW5rP7OvESdfaNTzkOgaRiqBU6doeTzJepIKEjd1S_B4Z8gS5jfyRCK8PHcQNEdqHRTmaEmGfTdVIYdom_vQMuz9hVaOpEbohDxBfHPXzVql07K2MsYH8vRcIWsbO1OiEfK5-COJTizF1XJIjrMhP4d0eQneFwE-UP3ErZLX7_ZQRxlfm83YMF5yyBiO4R-XNOx7Comment #41
catch@mherchel can you check if drush cr deletes the aggregate file from disk? And can you try a hard browser refresh to see if you get the new contents from it once it's recreated?
We might want to add the css_js_query_string back to libary URLs (as a separate query parameter, not part of the hash) to ensure the new version gets loaded, but would be its own issue.
Comment #42
mherchelYep. Verified its working 🙌
Testing steps
Thank you!
Comment #43
lauriiiShouldn't we keep this?
Comment #44
catch@lauriii it's no longer set by anything non-deprecated in core so it's redundant code in the new classes.
Comment #45
catchDiscussed #43/#44 with @lauriii - while the delete should be a no-op, in the interests of keeping changes to the absolute minimum in 10.1, adding it back.
Comment #48
lauriiiCommitted 405f8b33a7 and pushed to 11.0.x. Also committed the 10.1.x patch. Thanks!
Comment #49
badwebcoder commented#45, It works when I clear the cache via the admin toolbar, but not when I do it via Drush.
When I run
drush crI get the following messages:CSS and JS are not removed and new files are not created.
Comment #50
badwebcoder commented#45, works when I run cache clear via admin toolbar, but doesn't work via Drush.
When I run
drush crcommand I get the following messages:As a result, old CSS and JS files are not deleted, and new ones are not created.
Sorry for the duplicate comment, i accidentally
Comment #51
catch@BadWebCoder please open a new issue. The existing code was already deleting files in that directory, just not the directory itself, so if drush isn't able to find the directory now, it probably wasn't able to find it beforehand either.
Comment #52
chikeI did get the notices @BadWebCoder mentioned when I cleared cache with Drush,
But when I refreshed the page, my CSS changes did apply to the page.
Off-topic: I can also see that some of the forum topics I added on the site which I saw on the PC (it is a fresh install), viewing the site on a phone I can't see the most recent forum topics. Content this time and not CSS or JS.
Comment #53
chikeKindly ignore the off-topic above. It was my fault. I had wrong forum access permissions set.
Comment #54
chikeIf CSS aggregation is turned on, CSS changes don't show up on the site and the site remains in the past never changing, whether caches are cleared in Drush or UI.
Comment #55
chi commented@catch, seems it is deleting the directory as well.
https://git.drupalcode.org/project/drupal/-/blob/10.1.1/core/lib/Drupal/...
Comment #56
finex commentedCurrently it's also deleted the JS directory and both are never recreated (tested on both 10.1.1 and 10.1.x-dev).
Comment #57
catchMissed a bit #3376927: Remove even more of the aggregate stale file threshold and state entry.
Comment #59
edmund.dunn commentedWe are having the same issue as @FiNeX in #56.
Comment #60
quietone commentedpublished the change record
Comment #61
wim leersAFAICT this incorrectly removed
system.performance:stale_file_thresholdfrom config schema. It should have deprecated that instead? 🤔 (That is possible since #2997100: Introduce a way to deprecate config schemas.)