I just installed the CDN module, and it does an excellent job of any Drupal site to use a CDN. Some of what a CDN does (serving static files without firing up PHP) is already handled by Barracuda, but two other key features of CDNs are: 1. offloading of traffic and 2. speeding up file reception by serving them from the edge location nearest to the requester.

In the CDN module, there's a setting that gzips static files (as appropriate) and gives them far-future expiration dates. Running this module with Barracuda, when you leave this setting turned off, the files are served by the CDN, but the speed-gains are offset by the lack of compression & far-future expirations. However, if you turn this setting on in Barracuda, no CSS or javascript files are served at all for the site. I filed a support request on the CDN module (see here: #1724000: Can I use far-future expiration on omega8's nginx setup? ) and the response basically tells me that this can be fixed with some changes to the nginx configuration.

I think support for this module fits in perfectly with the Barracuda/Octopus focus on fast sites with low resource usage. And since I don't know very much about nginx config, I thought I'd put the feature request in here.

Thanks for considering it.

-Joseph

Comments

jtbayly’s picture

In researching this, I also found this little tidbit that could be helpful: http://www.nomitor.com/blog/2010/11/10/gzip-support-for-amazon-web-servi...

According to that link, in order for Nginx to support Amazon Cloudfront delivering gzipped content, the gzip_http_version needs to be set to 1.0. Now that was 2 years ago, so it could have changed, but I thought it was worth noting here, just in case.

-Joseph

omega8cc’s picture

Project: Barracuda » Octopus
Status: Active » Needs review

Here is completely untested config you could add in the extra include file, as explained in the docs:

###
### CDN Far Future expiration support.
###
location ^~ /cdn/farfuture/ {
  location ~* ^/cdn/farfuture/ {
    access_log off;
    expires    epoch;
    limit_conn gulag 88;
    rewrite    ^/cdn/farfuture/(?:[0-9a-zA-Z])*/(.+)$ /$2 last;
    add_header ETag "";
    add_header Vary "Accept-Encoding";
    add_header Accept-Ranges "bytes";
    add_header Cache-Control "private, must-revalidate, proxy-revalidate";
    add_header X-Header "CDN Generator 1.0";
    set $nocache_details "Skip";
    try_files  $uri @drupal;
    location ~* \.(?:css|js|svg|ico|gif|jpg|jpeg|png|otf|ttf|eot|woff|flv|swf)$ {
      add_header Cache-Control "max-age=290304000, no-transform, public";
      add_header Last-Modified "Wed, 20 Jan 1988 04:20:42 GMT";
      expires    max;
    }
  }
}

Could you test it and see if that helps?

omega8cc’s picture

omega8cc’s picture

This experimental config has been committed to BOA HEAD: http://drupalcode.org/project/barracuda.git/commit/ec7577c

jtbayly’s picture

Status: Needs review » Needs work

Sweet deal! Thanks for working on this.

I tried using it, but there were two problems. First, the rewrite wasn't quite right, but I think I managed to fix it. There are actually two "directories" added after "farfuture", not just one. Also, they weren't just alphanumeric, but also could contain some symbols. I was able to come up with two examples of different types of generated URLs that should be forwarded. Here is an example of each:

http://my.domain.com/cdn/farfuture/x5E_sG2mohSfzRmYQncvKqY2Nfc3UHdifrybk5HaR7Vs/mtime:43444553996/sites/my.domain.com/files/js/js_-ee4lnX3FpHqxxULJlddn_OVc29Ghc7WfGJL23OaFI0.js
http://my.domain.com/cdn/farfuture/gilZCzQfySRkP1HNe5Xndvw9amSD9yFPSjWJh0hni8/drupal:7.14/misc/feed.png

And here is the new rewrite that I believe works for both kinds:
rewrite ^/cdn/farfuture/[^/]+/[^/]+/(.+)$ /$1 last;

But the other problem is that the third "location" block (with css|js|etc) isn't working right. It is generating 404 errors. If I remove it, the 404 errors go away, but obviously the headers don't get added. Any idea how to fix it? I tried changing the rewrite flag above from "last" to "break" but it didn't make a difference. (I couldn't fully grok the documentation for nginx rewrite flags, but it seemed like we might want "break".) I also tried moving the try_files line after the block and inside the block, but I was just taking stabs in the dark at that point, so I thought I'd come back here for more help.

Thanks again,
-Joseph

jtbayly’s picture

Just to be clear, that means I'm using the following config:

###
### CDN Far Future expiration support.
###
location ^~ /cdn/farfuture/ {
  location ~* ^/cdn/farfuture/ {
    access_log off;
    expires    epoch;
    limit_conn gulag 88;
    rewrite    ^/cdn/farfuture/[^/]+/[^/]+/(.+)$ /$1 last;
    add_header ETag "";
    add_header Vary "Accept-Encoding";
    add_header Accept-Ranges "bytes";
    add_header Cache-Control "private, must-revalidate, proxy-revalidate";
    add_header X-Header "CDN Generator 1.0";
    set $nocache_details "Skip";
    try_files  $uri @drupal;
  }
}

This is the part I had to remove:

    location ~* \.(?:css|js|svg|ico|gif|jpg|jpeg|png|otf|ttf|eot|woff|flv|swf)$ {
      add_header Cache-Control "max-age=290304000, no-transform, public";
      add_header Last-Modified "Wed, 20 Jan 1988 04:20:42 GMT";
      expires    max;
    }

I still haven't been able to figure out how to get it to work. I just get 404 errors when I put it back in. Looking at the nginx documentation, I can't figure out why. I'm sure I'm missing something obvious...

It actually seems to do a pretty good job, even without it, but obviously it would be better if we could get that part working too.

-Joseph

omega8cc’s picture

Status: Needs work » Needs review

Hmm, interesting. I have copied the regex for this rewrite as-is from the README:

+  # Transform /cdn/farfuture/***/sites/default/files to /sites/default/files
+  RewriteCond %{REQUEST_URI} ^/cdn/farfuture/([0-9a-zA-Z])*/(.+)$
+  RewriteRule .* /%2 [L,E=FARFUTURE_CDN:1]

Here is the updated version - I didn't test it yet, because we have higher priority updates and fixes to include in the release this weekend.

Could you test and let us know if/how it works?

###
### CDN Far Future expiration support.
###
location ^~ /cdn/farfuture/ {
  access_log off;
  log_not_found off;
  limit_conn gulag 88;
  add_header ETag "";
  add_header Vary "Accept-Encoding";
  add_header Accept-Ranges "bytes";
  add_header X-Header "CDN Generator 1.0";
  set $nocache_details "Skip";
  location ~* ^/cdn/farfuture/.+\.(?:css|js|jpe?g|gif|png|ico|bmp|svg|swf|pdf|docx?|xlsx?|pptx?|tiff?|txt|rtf|class|otf|ttf|woff|eot|less)$ {
    expires    max;
    add_header Cache-Control "max-age=290304000, no-transform, public";
    add_header Last-Modified "Wed, 20 Jan 1988 04:20:42 GMT";
    try_files $uri @nobots;
  }
  location ~* ^/cdn/farfuture/ {
    expires    epoch;
    rewrite    ^/cdn/farfuture/[^/]+/[^/]+/(.+)$ /$1 last;
    add_header Cache-Control "private, must-revalidate, proxy-revalidate";
    try_files $uri @nobots;
  }
  try_files $uri @nobots;
}
Wim Leers’s picture

omega8cc’s picture

Thanks! I will submit Nginx version there.

omega8cc’s picture

jtbayly’s picture

LOL. Nope, I can't test it anymore, since we moved all our sites onto your hosting. :)

Actually, don't you run all the sites on your servers on HEAD? Maybe I could do testing still... Let me know.

-Joseph

omega8cc’s picture

Ah, that is a good news! :)

We do use head on many (not all) servers, but this latest update is not included on any system we host, yet. It will be after this weekend: #1816100: BOA-2.0.4 release - update

If it will still not work as expected, please let us know, as we can always add custom overrides for Nginx config to test improvements without upgrading the system.

omega8cc’s picture

Status: Needs review » Fixed

We have added/tested some fixes and now it works (in HEAD) - at least we are using it with the farfuture enabled on our own website. If others could confirm that it is implemented correctly, it would be great.

Related commits:

http://drupalcode.org/project/octopus.git/commit/408e74e
http://drupalcode.org/project/octopus.git/commit/5e9b08b

We will post dedicated article to explain correct configuration and integration with local (fake) CDN.

Wim Leers’s picture

Awesome :)

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.