Rewrite rules for gzipped CSS and JavaScript aggregates cause lots of lstats for files that will never exist

This seems interesting to me.
subscribe

subscribe

Good idea to write that .htaccess file dynamically into the appropriate folders, if enabled. And yes, let's make that configurable.

I did some server side benchmarks on http://drupal.org/node/101227#comment-3176382 - mod_deflate came out over 10x slower, and the file stats were not detectable (they might be on NFS, of course). I am not sure the CPU overhead of mod_deflate is favorable compared to the file stats on the vast majority of setups. I should also point out that if you care about file stats, you should be using vhost configuration rather than .htaccess in the first place.

Tweaking the regexp here so it doesn't look for js.gz.gz sounds like a fine idea, as does dynamically writing the .htaccess to the files/css and files/js directories. The only downside of this is that I was thinking that it might be nice to write a little Drush script to make gzip versions of all the contrib js and css, and the latter change would make this harder (you would need to copy all the files to files/css and files/js and write a module to alter all the requests to this directory).

subscribe

I'm tempted to suggest this whole feature be rolled back - not clear it's worth the headache for a typical site.

@pwolanin - see http://drupal.org/node/101227#comment-3152752 "this patch gives a ~20-60% saving in page load performance for initial page loads (or other page loads where you download an aggregate), saving somewhere from ~0.3-3 seconds, depending on your connection speed". I think this is *well* worth the headache - this is probably the single biggest measurable improvement in end user performance in Drupal 7.

File stats were not measurable in my tests, the IO gain from having to read/serve a smaller file in the first place may actually compensate completely. Sites with NFS storage are hardly that typical, it has not yet been shown that mod_deflate is actually faster even using NFS - and in any case this is trivial to remove from htaccess if this is a concern. The proposed changes are reasonable performance tweaks - but I don't see any fundamental problem here.

I'm with Owen Barton on this one. Let's try to get rid of the .gz.gz stat, but in general this is a very good performance improvement for the vast majority of drupal sites out there.

As for the performance implications of a stat, I found one interesting article. In this article the author says that using a stat cache "just got in the way"
http://www.sirgroane.net/2010/03/tuning-glusterfs-for-apache-on-ec2/

Frequently used files will get cached in memory. I don't know about a cache miss, because the file isn't there, and how that will effect performance on a shared file system. This is something that needs to be benchmarked. For the the vast majority of drupal sites out there, they are on one box, this is a win; that's what Owen Barton's benchmarks showed. Multi webhead setups need to be tested before we make a decision on this; but because of drupal's market share with single server setups I am against reverting this change.

Here's how core does "dynamic" htaccess writing.
http://api.drupal.org/api/drupal/includes--file.inc/function/file_create...

Looks like we would have to do an htaccess check every time a .gz file is created in the css/js dir drupal_build_css_cache, drupal_build_js_cache. Just like file_unmanaged_copy does the check every time a file is created in the files directory.

Fixed this issue tag's spelling.

Untested; but worth testing out IMHO. RewriteCond %{REQUEST_URI} (^/(.*)(\.js|\.css)) [NC] comes from the parallel module; forget why I didn't use RewriteCond %{REQUEST_URI} (\.js|\.css)$ [NC] maybe it works; I forget.

  <IfModule mod_headers.c>
    # Serve gzip compressed JS/CSS files if they exist and the client accepts gzip.
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_URI} (^/(.*)(\.js|\.css)) [NC]
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*) $1\.gz [QSA]

    # Serve correct content types, and prevent mod_deflate double gzip.
    RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
    RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]

    <FilesMatch "(\.js\.gz|\.css\.gz)$">
      # Serve correct encoding type.
      Header append Content-Encoding gzip
      # Force proxies to cache gzipped & non-gzipped css/js files separately.
      Header append Vary Accept-Encoding
    </FilesMatch>
  </IfModule>
</IfModule>

Another alt

  <IfModule mod_headers.c>
    # Serve gzip compressed JS/CSS files if they exist and the client accepts gzip.
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_URI} (^/(.*)/(css|js)/(.*)(\.js|\.css)) [NC]
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*) $1\.gz [QSA]

    # Serve correct content types, and prevent mod_deflate double gzip.
    RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
    RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]

    <FilesMatch "(\.js\.gz|\.css\.gz)$">
      # Serve correct encoding type.
      Header append Content-Encoding gzip
      # Force proxies to cache gzipped & non-gzipped css/js files separately.
      Header append Vary Accept-Encoding
    </FilesMatch>
  </IfModule>
</IfModule>

It must be 5 years since I used apache so my mod_rewrite skills are not the best.

RewriteCond %{REQUEST_URI} (^/(.*)/(css|js)/(.*)(\.js|\.css)) [NC]

could it be ...

RewriteCond %{REQUEST_URI} ^/.*/(css|js)/.*\.(css|js)$ [NC]

and

<FilesMatch "(\.js\.gz|\.css\.gz)$">

could be

<FilesMatch "\.(js|css)\.gz$">

? Other than that with a quick refresher through the apache docs it looks like it would work fine. Course Nginx and the gzip_static module takes care of all of that functionality anyway ;-)

As a side note I'm a bit surprised Drupal 7 assumes the .htaccess files will just work, while they are by default disabled in *every* Aegir install and are never used in any Nginx based install.

Is it possible to simply disable creating those never used files (and checks if they exist)? Am I missing something? It looks like a waste of resources (php + I/O) for any site not using Apache or managed in the Aegir Hosting System?

Commenting on the issue of file stats, if you open a file with the intention to gzip it, then at os level it will perform a stat to check the existance of the file before attempting to open it anyway, then there is IO to read that file, cpu to crank out the output, if the file is large apache may write the gzip output into and temporary file etc etc.
When I first started experimenting with boost I looked into the performance cost of file stats and after a fair bit of fannying around I realised that the cost is pretty negligable.
As mentioned earlier in the post if there are 10+ files to compress per page I would definately say that 10 file stats would be quicker.
The point about misc/(js|css) files is a good one though, along with having the .htaccess files in the directory, although if memory serves me right you need a special directive in apache to get it to look for .htaccess files in all directories... oh no more file stats to check for the .htaccess files in each directory... where will it end!!! ;-)

Also, to speed up the regex it is a good idea to disable back-references when they are not used to match/replace anything, so in this case:

RewriteCond %{REQUEST_URI} ^/.*/(?:css|js)/.*\.(?:css|js)$ [NC]

At least it works for regex in Nginx, no idea if that is compatible with Apache, but it should be, imo.

Oh, and while we are at caching files checks, it is so easy in Nginx with something like:

  open_file_cache max=8000 inactive=30s; 
  open_file_cache_valid          60s; 
  open_file_cache_min_uses         3;
  open_file_cache_errors          on;

subscribe

I hit a problem with these rewrite rules. Let's say I have http://example.com and http://hosting.com/example.com which lead to the same physical path with Drupal 7. example.com works perfectly. I had Alias /example.com /path/to/drupal and when hosting.com/example.com is accessed, Apache's rewrite engine mysteriously transforms the new URI /sites/...css into /path/to/drupal/sites/...css and handles it like URI, resulting in Not Found error. I removed the Alias directive and put symbolic link instead and everything works perfectly. There's something with Apache's order of URI-to-filename handling and it breaks in this case.

There has to be some Drupal docs that warns that Alias and these rewrites don't play together.

issues I've encountered with boost and apache alias
#1044814: Boost working with a multisite subfolder installation (not subdomain)
Hard coding the document root is how I got around the issue. symlinks are 100% better.

Gone through the rules in some detail (at least from a theoretical regex point of view - I haven't tried to profile the code) and come up with the following patch.

The separate js/css rewrite blocks are replaced by one. I opted to leave the extension check in the rewrite - moving it to a separate RewriteCond doesn't add any performance that I can see (any complexity removed by the rewrite is added to the cond, and you have another setup/teardown of the regex matcher too).

Both the rewrite and FilesMatch regular expressions have been optimized to limit the number of character matches in the parens - \.(j|cs)s\.gz - this should avoid backtracks in some cases. Although it is a good idea, I didn't add any non-capturing parens, as they seem to really only apply to the FilesMatch (which may disable them itself) and are only supported in Apache 2 (and I suspect may break Apache 1).

I think the next step is to move this block into php such that it is written to files/css and files/js dynamically, as Catch describes.

Escaping dots (RE-style) in the second argument of RewriteRule is useless. The only special characters in substitution are $ and %.

RewriteRule ^(.*)\.(j|cs)s$ $1\.$2s\.gz [QSA]

  <FilesMatch "(\.js\.gz|\.css\.gz)$">
      # Serve correct encoding type.
      Header append Content-Encoding gzip
      # Force proxies to cache gzipped & non-gzipped css/js files separately.
      Header append Vary Accept-Encoding
    </FilesMatch>

Wonder why it's use append rather that set or merge in serving the correct encoding type, I check for documentation here - http://httpd.apache.org/docs/current/mod/mod_headers.html#examples

The problem I've encountered in Header append Content-Enocding gzip is I can't enable CSS and JS aggregation in Drupal performance page, the site breaks if it is installed in a Apache server which already doing gzip compression, the gzip in the header sent to the browser seems to be twice, please take a look in the attached screenshot.

This happens when Drupal is installed in a sub domain, sub directory as well as in multi-site setup using symlink, I've tested it in those situation. Drupal installed in the root directory seems not affected, I wonder why but if you want to verify this problem, here I setup a fresh install of a D7 site - > http://d7.levinson.com.ph

Login with user : admin/admin
go to configuration > performance
then turn on CSS / JS aggregation
VIOLA - the site is broken!

No effects even if you clear the cache several times.

Changing the code

      # Serve correct encoding type.
      Header append Content-Encoding gzip

to

      # Serve correct encoding type.
      Header set Content-Encoding gzip

seems to solve the problem.

Any suggestion on this?

I can confirm the double encoding on that site. Saving one of the aggregated css files as a .css.gz and opening it has the correct contents then.

I'll be changing advagg from append to set as a result of this test.

Please open a new issue for the double encoding header and post the issue # here - we can probably backport that change to D7

Here's the new issue - > http://drupal.org/node/1116416

I came across this page

http://www.webmasterworld.com/apache/4217718.htm

where 2 .htaccess cracks (one of them they claim rewrote the Wordpress one) work on the drupal .htaccess file.

One of the parts they rewrite is the .gz part, and they make it look like

# Rules to serve gzip-compressed CSS and JS files.
# Requires both mod_rewrite and mod_headers to be enabled.
#
# If the client accepts gzip and compressed CSS and JS files exist, serve them with
# proper Content-Type headers and set the no-gzip variable to prevent double-encoding
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -s
RewriteCond $2->text/css ^css->(.+)$ [OR]
RewriteCond $2->text/javascript ^js->(.+)$
RewriteRule ^(.+\.(css|js))$ /$1.gz [T=%1,E=no-gzip:1,L]

Adding

And that <Filesmatch> on the Content-Encoding section could also be shortened-up...

<FilesMatch "\.(css|js)\.gz$">

As most people here, apache .htaccess files are not my forte either, so using knowledge of people who do understand this seems a good idea. The thread on webmasterworld is pretty old, but you could always try revive it (referring here), i guess.

This looks like an improvement to be sure (although it further reduces the number of people who grok that piece of code). I think the main thing to do would be to test it, to see if it does indeed reduce the number of fstat calls and of course ensuring headers are all correct/unchanged with all the different combinations (Apache 1 & 2, each module enabled/disabled, .gz files present/absent).

(the slash in the i/o tag breaks the autocomplete from adding new tags)

There is another error in the .htaccess that will be solved by this patch: json files that happen to have a json.gz version as well, will be rewritten to ....js.gz. If people think that this error needs a backport to D7, I will file it as a separate issue.

Added a release note snippet

I read through this issue and the original issue - #101227: Gzip aggregated CSS and JS. What seems odd to me is that the apache docs for mod_deflate say:

Since mod_deflate re-compresses content each time a request is made, some performance benefit can be derived by pre-compressing the content and telling mod_deflate to serve them without re-compressing them.

And the original issue has performance tests that show that performance is improved by Drupal's current approach as opposed to mod_deflate.

There's nothing to stop a site from disabling compression and changing their .htaccess currently. It's just what should the default be. Does the current configuration save bandwidth costs for those on shared / smaller hosting? Will this change be to their detriment. I think we need some analysis to refute the findings of #101227-122: Gzip aggregated CSS and JS

Regex looks good to me.

I think this is only for 10.0.x - right?

Oh yeah .htaccess changes need a release note, I think.

Can we move the details including the regex stuff into a CR and link it ? Thanks!

I would suggest something like:

The htaccess rewrite rule that enables static gzipped JavaScript and CSS aggregates to be served to browsers has been modified to improve performance. Sites should <a href="">update their .htaccess files</a> to take advantage of this performance improvement.

Saving issue credit.

Discussed with @catch we agreed to backport to 9.5.x as this is a bugfix and although .htaccess is fixed applying the changes is not 100% necessary - though it is good for performance.

Committed and pushed a6c49dbae4 to 10.1.x and 3ef4faacbb to 10.0.x and 8412918b4d to 9.5.x. Thanks!

Closed #2451003: Apache fails to serve gzip files if name contains .css or .js as a duplicate, adding credit.