Addressing Nginx support for advagg - mostly in 404 handling [#1169702]

Mike, when I did apply the full Drupal method outlined for advagg's Drupal based 404's, I got slower (and skewered) response times for most static assets pulled via advagg's aggregation through a CDN. This given Drupal 404 method handling certainly made the advagg 'report' status happy - but real life benchmarks indicated I had a 40% slower response to static asset requests.

I went back to do handling this the 'previous' Nginx way (fast selected 404's that Drupal only will only ever see for for 2 chosen locations -> imagecache + advagg + more as required).

Testing: Benchmark testing indicated that the 'lost' 40% performance was restored. However, 'advagg' does complain - about my setup not being so 'cool'. (Is this a legacy of the Apache piglet syndrome coding factor?)

So, for me, after days spent on a variety of mod's for this advagg aggregation testing, I am about to hack out (grin) advagg's 'report' reasoning on 404 drupal handling. I suspect it was reasoned more for Apache +Drupal + .htaccess logic. For despite all the report 'complaints' issued by advagg reasoning - it works fast - with nginx + php-fpm. I find I have forgotten most of the 'Apache methods' I once needed and coded (2 years ago)!

Have you thought about some serious Nginx compat? I am....(have)

Great project by the way! Just kindly allow some of your good patience to extend those D6's that have moved the RU 'nginx' Drupal way. I think there is more than just me?

Comment	File	Size	Author
#30	advagg-1169702-30.patch	12.58 KB	mikeytown2
#18	debug.png	76.64 KB	mikeytown2
#5	advagg-1169702-5.patch	3.63 KB	mikeytown2

Comments

Comment #1

mikeytown2 commented 26 May 2011 at 17:16

Status:

Active

» Closed (duplicate)

#1116618: Need a nginx setup doc

People have gotten this working with nginx.

Comment #2

Peter Bowey commented 26 May 2011 at 23:12

Works fine for nginx Mike. Just concerned with the advagg 'status report values' of doing so....

Comment #3

mikeytown2 commented 27 May 2011 at 02:48

Status:

Closed (duplicate)

» Active

Run this code as php input and let me know what you get

  global $base_path;

  list($css_path, $js_path) = advagg_get_root_files_dir();
    $ip = variable_get('advagg_server_addr', FALSE);
    if ($ip == -1) {
      $ip = $_SERVER['HTTP_HOST'];
    }
    elseif (empty($ip)) {
      $ip = empty($_SERVER['SERVER_ADDR']) ? '127.0.0.1' : $_SERVER['SERVER_ADDR'];
    }
  $filepath = $css_path . '/css_missing' . mt_rand() . time() . '_0.css';
  $url = 'http://' . $ip . $base_path . $filepath;
  $headers = array(
    'Host' => $_SERVER['HTTP_HOST'],
  );

  $data = drupal_http_request($url, $headers);
  echo $url . "<br>\n" . str_replace('    ', '&nbsp;&nbsp;&nbsp;&nbsp;', nl2br(htmlentities(print_r($data, TRUE))));

You should have this for data

    [data] => <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL was not found on this server.</p><p><a href="/">Home</a></p><!-- advagg_missing_fast404 --></body></html>

And this should be in the headers

[X-AdvAgg] => Failed Validation. Wrong Pattern.

Comment #4

mikeytown2 commented 27 May 2011 at 02:50

What I should do is have admin/settings/advagg/info display this information for you in one of the fieldsets.

Comment #5

mikeytown2 commented 27 May 2011 at 03:22

Status	File	Size
new	advagg-1169702-5.patch	3.63 KB

with this patch you can go to "admin/settings/advagg/info" select "Asynchronous debug info" and it should output the correct debugging info. Patch has been committed.

Comment #6

Peter Bowey commented 27 May 2011 at 04:33

Refer #3, #4, and #5
Thanks Mike, *much appreciated* - will help work through this... (later)

For the moment, a 'heads up' on a related [nginx] issue; when I run advagg rebuild, I note that it rapidly opens a lot of http connections (probally normal). I get nginx warnings that 'hit over' my current nginx domain setting 'max connection limit' [gulag]; eg [nginx.conf]

limit_conn          gulag 12;                                           # max concurrent connections per client /ip

Captured nginx logs during rebuild:

[error] *15 limiting connections by zone "gulag", client: 192.168.0.1, server: www.peterbowey.com.au, 
request: "GET /sites/peterbowey.com.au/files/advagg_css/css_c2c64ebbd07e2c188b7dbb6ced8d3113_1.css?generator=1 HTTP/1.0", host: "www.peterbowey.com.au"

[error] *14 limiting connections by zone "gulag", client: 192.168.0.1, server: www.peterbowey.com.au, 
request: "GET /sites/peterbowey.com.au/files/advagg_css/css_128a5c9bf70a7e529bf5ea45b347a55a_1.css?generator=1 HTTP/1.0", host: "www.peterbowey.com.au"

[error] *13 limiting connections by zone "gulag", client: 192.168.0.1, server: www.peterbowey.com.au, 
request: "GET /sites/peterbowey.com.au/files/advagg_css/css_3b89b7340827905c6f3f823511332543_1.css?generator=1 HTTP/1.0", host: "www.peterbowey.com.au"

I am 'bashing away' at improving this nginx<->advagg marriage in between actual work projects...

... 'more to come' ...

Comment #7

Peter Bowey commented 27 May 2011 at 04:44

referring #6
Just had a rather large 'advagg rebuild' speed improvement = 185% [cool]

I increased the nginx 'max connections limit'; [limit_conn gulag 20;]
[Conclusion:] advagg rebuild does open a lot of async' http connections! That is good!

I just did not expect it,... hmm what about DdoS limits.... OK there are other ways....

Comment #8

Peter Bowey commented 27 May 2011 at 04:49

By heck, the *more http connections* I allow, the faster goes the advagg rebuild.... +400%
Truly we have a async http, now for some DdoS limits.

Comment #9

Peter Bowey commented 27 May 2011 at 06:51

Report for #5

stdClass Object (
    [request] => GET /sites/peterbowey.com.au/files/advagg_css/css_missing376173171306476907_0.css HTTP/1.0
Host: www.peterbowey.com.au
User-Agent: Drupal (+http://drupal.org/)

    [data] => <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
    <head>
        <title>The page is not found</title>

Notes 1: I note that this is nginx's '404' response and not what should be sent to Drupal! Will investigate!

Notes 2: My commercial CDN [Edgecast] strips out advagg's header append [X-Advagg] + !

Above is 'More or less proven' by routing to my [local] own CDN.
Will now 'code' around this problem...

Comment #10

Peter Bowey commented 27 May 2011 at 07:22

OK, (change = no Nginx 404 handler) we now have the correct 'advagg debug' report:

stdClass Object (
    [request] => GET /sites/peterbowey.com.au/files/advagg_css/css_missing17397375241306480456_0.css HTTP/1.0
Host: www.peterbowey.com.au
User-Agent: Drupal (+http://drupal.org/)

    [data] => <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL was not found on this server.</p><p><a href="/">Home</a></p><!-- advagg_missing_fast404 --></body></html>
    [protocol] => HTTP/1.1
    [status_message] => Not Found
    [headers] => Array (
            [Server] => nginx
            [Date] => Fri, 27 May 2011 07:14:15 GMT
            [Content-Type] => text/html; charset=utf-8
            [Connection] => close
            [X-Accel-Expires] => 0
            [X-Drupal-Cache] => MISS
            [Last-Modified] => Fri, 27 May 2011 07:14:16 +0000
            [ETag] => "1306480456"
            [X-AdvAgg] => Failed Validation. Wrong Pattern.
            [Expires] => Sun, 19 Nov 1978 05:00:00 GMT
            [Cache-Control] => must-revalidate
            [Content-Length] => 307
        )
    [error] => Not Found
    [code] => 404
    [timer] => Array (
            [count] => 1
            [time] => 58.7
        )
)

However, advagg reports this status (which is the concern that I have been chasing):

Current State: Set to TRUE. Check your CDN settings; request is not coming back when routed through the CDN.

Comment #11

mikeytown2 commented 27 May 2011 at 07:40

What does the CDN debug output look like?

Comment #12

Peter Bowey commented 27 May 2011 at 08:06

Have not done CDN debug yet?

However, I have noted this problem doing it 'the Drupal 404 way' (for advagg):

Request 3:

URL: http://small.gdlcdn.com/802C5F/cdn1/sites/peterbowey.com.au/files/advagg_css/css_feacfcb2dabf07bbd71c367937fb0069_0.css
Host: small.gdlcdn.com
IP: 72.21.91.19
Location: Santa Monica, CA*
Error/Status Code: 500
Start Offset: 0.963 s
Initial Connection: 57 ms
Time to First Byte: 697 ms
Content Download: 0 ms
Bytes In (downloaded): 1.0 KB
Bytes Out (uploaded): 0.4 KB

Request Headers:

GET /802C5F/cdn1/sites/peterbowey.com.au/files/advagg_css/css_feacfcb2dabf07bbd71c367937fb0069_0.css HTTP/1.1
Accept: */*
Referer: http://www.peterbowey.com.au/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; PTST 2.285)
Accept-Encoding: gzip, deflate
Host: small.gdlcdn.com
Connection: Keep-Alive

Response Headers:

HTTP/1.1 500 Internal Server Error
Content-Type: text/html; charset=utf-8
Date: Fri, 27 May 2011 07:52:12 GMT
Server: nginx
Content-Length: 3693

A short 'wait' and the above 500 error becomes a live static asset.
This does not occur unless I 'give in' to the Drupal 404 method.

OK, which CDN 'debug' are you referring too?

Comment #13

mikeytown2 commented 27 May 2011 at 08:11

It should be below the normal one on the info page with he CDN url

Comment #14

Peter Bowey commented 27 May 2011 at 08:29

Refer to #12

The above nginx 500 error' shows a stream of errors like this in [nginx error log]:

2011/05/27 17:21:46 [error] 3867#0: *173 could not find named location "@drupal", client: 72.21.83.46, server: cdn1.peterbowey.com.au, request: "GET /sites/peterbowey.com.au/files/advagg_css/css_feacfcb2dabf0
7bbd71c367937fb0069_0.css HTTP/1.1", host: "cdn1.peterbowey.com.au", referrer: "http://www.peterbowey.com.au/"
2011/05/27 17:21:46 [error] 3867#0: *177 could not find named location "@drupal", client: 72.21.83.62, server: cdn1.peterbowey.com.au, request: "GET /sites/peterbowey.com.au/files/advagg_css/css_3b89b73408279
05c6f3f823511332543_0.css HTTP/1.1", host: "cdn1.peterbowey.com.au", referrer: "http://www.peterbowey.com.au/"
2011/05/27 17:22:04 [error] 3862#0: *187 could not find named location "@drupal", client: 72.21.83.46, server: cdn1.peterbowey.com.au, request: "GET /sites/peterbowey.com.au/files/advagg_css/css_feacfcb2dabf0
7bbd71c367937fb0069_0.css HTTP/1.1", host: "cdn1.peterbowey.com.au", referrer: "http://www.peterbowey.com.au/"
2011/05/27 17:22:04 [error] 3862#0: *188 could not find named location "@drupal", client: 72.21.83.62, server: cdn1.peterbowey.com.au, request: "GET /sites/peterbowey.com.au/files/advagg_css/css_3b89b73408279
05c6f3f823511332543_0.css HTTP/1.1", host: "cdn1.peterbowey.com.au", referrer: "http://www.peterbowey.com.au/"

Will investigate my nginx.conf settings :)

Comment #15

Peter Bowey commented 27 May 2011 at 08:29

Refer to #11

CDN integration statistics for node

    Total number of files on this page: 51.
    Number of files available on CDNs: 51 (100% coverage).
    Number of files served from the server http://small.gdlcdn.com/802C5F/cdn3: 8
    Number of files served from the server http://small.gdlcdn.com/802C5F/cdn1: 30
    Number of files served from the server http://small.gdlcdn.com/802C5F/cdn2: 13
    Total time it took to look up the CDN URLs for these files: 1.756 ms, or 0.034 ms on average per file.
    The files that are synchronized to the CDN:
        sites/all/themes/litehouse/favicon.ico (server: http://small.gdlcdn.com/802C5F/cdn3)
        sites/peterbowey.com.au/files/imagecache/lead-image-teaser/06_0.jpg (server: http://small.gdlcdn.com/802C5F/cdn3)
        sites/peterbowey.com.au/files/imagecache/lead-image-teaser/04_1.jpg (server: http://small.gdlcdn.com/802C5F/cdn3)
        sites/all/themes/litehouse/images/feed-icon.png (server: http://small.gdlcdn.com/802C5F/cdn3)
        misc/blog.png (server: http://small.gdlcdn.com/802C5F/cdn3)
        sites/peterbowey.com.au/files/imagecache/carousel-image/04_2.jpg (server: http://small.gdlcdn.com/802C5F/cdn3)
        sites/peterbowey.com.au/files/imagecache/carousel-image/05_0.jpg (server: http://small.gdlcdn.com/802C5F/cdn3)
        sites/peterbowey.com.au/files/imagecache/carousel-image/carousel_01.jpg (server: http://small.gdlcdn.com/802C5F/cdn3)
        modules/aggregator/aggregator.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/node/node.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/poll/poll.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/system/defaults.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/system/system.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/system/system-menus.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/user/user.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/cck/theme/content-module.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/ctools/css/ctools.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/filefield/filefield.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/lightbox2/css/lightbox.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        modules/forum/forum.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/cck/modules/fieldgroup/fieldgroup.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/views/css/views.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/cdn/cdn.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/viewscarousel/jquery.jcarousel.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/modules/viewscarousel/skins/ie7/skin.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/all/themes/litehouse/css/style.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        misc/jquery.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        misc/drupal.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/modules/lightbox2/js/lightbox.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/modules/memcache/memcache_admin/memcache.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/modules/viewscarousel/jquery.jcarousel.pack.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/modules/viewscarousel/viewscarousel.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/themes/litehouse/js/superfish.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/themes/litehouse/js/cufon-yui.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/all/themes/litehouse/js/Pilsen_Plakat_400-Pilsen_Plakat_400.font.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/peterbowey.com.au/files/advagg_css/css_93301ccf00f1014fcec53fd80147d589_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_feacfcb2dabf07bbd71c367937fb0069_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_f807d81476b2507b17d510c6c41327ce_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_49e580f69795811bf7e88773444492f8_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_7d483e9ac501ce6dc8cb0847d49a009f_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_c81b20952e682353b997da40991cfe4d_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_3b89b7340827905c6f3f823511332543_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_128a5c9bf70a7e529bf5ea45b347a55a_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_c2c64ebbd07e2c188b7dbb6ced8d3113_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_69793baa832068e44ce4cd8ca118c138_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_acc6a23f92cef3db145e84b2118fca28_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_css/css_17191a7f38a37952f82728714fe7eaed_0.css (server: http://small.gdlcdn.com/802C5F/cdn1)
        sites/peterbowey.com.au/files/advagg_js/js_c86bf4009fd10e567562612848225a57_0.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/peterbowey.com.au/files/advagg_js/js_6068297dcf689c22db55013770e422b9_0.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/peterbowey.com.au/files/advagg_js/js_fd780a97111bbbd6160e59f53e27eef1_0.js (server: http://small.gdlcdn.com/802C5F/cdn2)
        sites/peterbowey.com.au/files/advagg_js/js_91d625edac51e897b52084a584c242fd_0.js (server: http://small.gdlcdn.com/802C5F/cdn2)

Comment #16

mikeytown2 commented 27 May 2011 at 08:29

On admin/settings/cdn/details what mode are you running? Origin Pull or File Conveyor?

Comment #17

Peter Bowey commented 27 May 2011 at 08:33

I would hardly call that 'debug' option (for CDN).
It is more CDN stats (enable)

Comment #18

mikeytown2 commented 27 May 2011 at 08:46

Status	File	Size
new	debug.png	76.64 KB

There's a second text area below the first one; what does it say? It should have the CDN url.

See attachment for example

Comment #19

Peter Bowey commented 27 May 2011 at 11:29

Thank Mike!! :)

/sites/peterbowey.com.au/files/advagg_css/css_missing17245313031306495611_0.css: 
stdClass Object (
    [error] => missing schema
    [code] => -1002
    [timer] => Array (
            [count] => 1
            [time] => 0.04
        )
)

Interesting, compared to the above *.png sample???

Comment #20

Peter Bowey commented 27 May 2011 at 12:02

#19 looks like a return error for an incorrect URI - via drupal_http_request() in common.inc
Hmmm... looking....

includes/common.inc, line 762

<?php
function drupal_http_request($url, array $options = array()) {
  $result = new stdClass();

  // Parse the URL and make sure we can handle the schema.
  $uri = @parse_url($url);

  if ($uri == FALSE) {
    $result->error = 'unable to parse URL';
    $result->code = -1001;
    return $result;
  }

  if (!isset($uri['scheme'])) {
    $result->error = 'missing schema';
    $result->code = -1002;
    return $result;
  }

Grep search in advagg folder:

[root@server1 advagg]# grep -R 'drupal_http_request' *

advagg.admin.inc:  $data_local = drupal_http_request($url, $headers);
advagg.admin.inc:    $data_cdn = drupal_http_request($url_cdn);
advagg.install:  $data_local = drupal_http_request($url, $headers);
advagg.install:    $data_cdn = drupal_http_request($url_cdn);
advagg_js_compress/advagg_js_compress.module:    $results = drupal_http_request($url, $headers, 'POST', $query_string);
advagg.module:          drupal_http_request($url, $headers, 'GET');
advagg.module: * @see drupal_http_request
advagg.module: * @see drupal_http_request

Possible area I need to look at is: (in advagg.install)

  if (module_exists('cdn')) {
    $url_cdn = file_create_url($filepath);
    // Send request and also time it.
    timer_start(__FUNCTION__ . 'cdn');
    $data_cdn = drupal_http_request($url_cdn);
    $time_cdn = timer_stop(__FUNCTION__ . 'cdn');
    $mode = variable_get(CDN_MODE_VARIABLE, CDN_MODE_BASIC);
  }

Looking deeper.

advagg.admin.inc

  if (module_exists('cdn')) {
    $url_cdn = file_create_url($filepath);
    // Send request and also time it.
    timer_start(__FUNCTION__ . 'cdn');
    $data_cdn = drupal_http_request($url_cdn);
    $data_cdn->timer = timer_stop(__FUNCTION__ . 'cdn');
    list($data, $rows) = advagg_form_print_r($data_cdn);
    $form['async']['cdn'] = array(
      '#type'           => 'textarea',
      '#title'          => check_plain($url_cdn),
      '#default_value'  => $data,
      '#rows'           => min($rows+1, 50),
    );
  }

Notes to self: "drupal_http_request expects the location field to contain an absolute URI.
URL is supposed to be absolute.
This URL redirects twice ??
From HTTP/1.1 RFC:

Location: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.30

The Location response-header field is used to redirect the recipient to a location other than the Request-URI for completion of the request or identification of a new resource. [...] *The field value consists of a single absolute URI.*

Location = "Location" ":" absoluteURI
example:
Location: <a href="http://www.w3.org/pub/WWW/People.html" title="http://www.w3.org/pub/WWW/People.html">http://www.w3.org/pub/WWW/People.html</a>

Comment #21

Peter Bowey commented 27 May 2011 at 16:21

Step 2: Debugging the URI used in both advagg's [advagg.install] and [advagg.admin.inc]...

Logic: I placed 2 strategic PHP error_log() placements at the value in $url_cdn - before the function call to drupal_http_request($url_cdn). See code fragments above...

The value of $url_cdn (in both places) is: /sites/peterbowey.com.au/files/advagg_css/css_missing11714652981306508212_0.css

Debug Result: For both advagg.install and advagg.admin.inc, we have advagg offering a 'relative url' as a URI for the function drupal_http_request($url_cdn)? Double Duh?

Comment #22

Peter Bowey commented 27 May 2011 at 15:58

Step 3: addition to URI in #21

Solution is near = Joy!

I placed a temp. code edit / addition to 'force' the correct URI required for the advagg's [advagg.install file] and [advagg.admin.inc file] like so:

  if (module_exists('cdn')) {
    $url_cdn = file_create_url($filepath);
    // Send request and also time it.
    timer_start(__FUNCTION__ . 'cdn');
+   $url_cdn = 'http://small.gdlcdn.com/802C5F/cdn1' . $url_cdn;   // Added to correct required URI
    error_log("advagg.install report:" . $url_cdn, 0);             // <<-- a temp debug catch event 'helper' only!
    $data_cdn = drupal_http_request($url_cdn);

And finally, the advagg module all works as it should! :)

Now the final advagg 'CDN result debug' report is (below):

http://small.gdlcdn.com/802C5F/cdn1/sites/peterbowey.com.au/files/advagg_css/css_missing10478433591306510068_0.css: 

Host: www.peterbowey.com.au
User-Agent: Drupal (+http://drupal.org/)

    [data] => <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL was not found on this server.</p><p><a href="/">Home</a></p><!-- advagg_missing_fast404 --></body></html>
    [protocol] => HTTP/1.1
    [status_message] => Not Found
    [headers] => Array (
            [Server] => nginx
            [Date] => Fri, 27 May 2011 15:27:48 GMT
            [Content-Type] => text/html; charset=utf-8
            [Connection] => close
            [X-Accel-Expires] => 0
            [X-Drupal-Cache] => MISS
            [Last-Modified] => Fri, 27 May 2011 15:27:49 +0000
            [ETag] => "1306510069"
            [X-AdvAgg] => Failed Validation. Wrong Pattern.
            [Expires] => Sun, 19 Nov 1978 05:00:00 GMT
            [Cache-Control] => must-revalidate
            [Content-Length] => 307
        )
    [error] => Not Found
    [code] => 302
    [redirect_code] => 404
    [redirect_url] => http://www.peterbowey.com.au/index.php?q=sites/peterbowey.com.au/files/advagg_css/css_missing10478433591306510068_0.css
    [timer] => Array (
            [count] => 1
            [time] => 315.14
        )
)

'Sigh', advagg is now processing CDN use 'as it should' (great news), so now to trace through why the correct URI commission is not occurring for the advagg CDN. A call made with URI = 'relative' (rather than absolute).

Comment #23

Peter Bowey commented 27 May 2011 at 16:24

Aggregate Static fetch time results are now normal!

See: http://www.webpagetest.org/result/110527_9V_PWYJ/2/details/

Thanks Mike, for without your new advagg debug feature, I would have floundered for some days on this issue!

I tidied the final advagg code with same 'prefix CDN hack' I used earlier!

....
$baseURI = variable_get('my_cdn_css_base_url', '');
....

Comment #24

mikeytown2 commented 27 May 2011 at 17:25

Status:

Active

» Fixed

@peter bowey
If I where you, I would create a module that implements hook_file_url_alter(), see cdn.module for an example. I would have it weigh more than CDN (heaver then 10,000) so it runs after it; this way you can add in the prefix and not have to hack everywhere to get this working correctly with your CDN setup.

I'm going to mark this fixed.

Comment #25

Peter Bowey commented 28 May 2011 at 00:35

#24 'comment only' = leave as 'fixed' status!

Thanks Mike,

I am still 'dense' enough to remain curious why I needed to re-form this URI (patch).

My first CDN path mod (addition)., I can understand - but not this one (all above).

In this 'case' http://drupal.org/comment/edit/4527382, I am not just adding a 'prefix' - I am reforming a malformed URI (an absolute).

Where the 'heck' does advagg derive $url_cdn = 'relative'?

eg:$url_cdn = '/sites/peterbowey.com.au/files/advagg_css/css_missing...'

My CDN Module map is:

CDN mapping: (via CDN D6 module)
http://small.gdlcdn.com/802C5F/cdn1|.css
http://small.gdlcdn.com/802C5F/cdn2|.js
http://small.gdlcdn.com/802C5F/cdn3|.jpg .png .gif .ico
http://small.gdlcdn.com/802C5F/cdn4|.pdf

where relative??

Comment #26

mikeytown2 commented 28 May 2011 at 01:21

Not sure why your getting that.

$filepath = $css_path . '/css_missing' . mt_rand() . time() . '_0.css';
$url_cdn = file_create_url($filepath);

file_create_url() should run cdn_file_url_alter() which should detect that this is a file that should be CDN-ed and replace the relative path with the absolute path pointing to your CDN. As for why file_create_url() didn't work, I would look into your CDN settings and/or see if you hacked cdn_file_url_alter in a way that is preventing this from working.

Comment #27

mikeytown2 commented 28 May 2011 at 01:23

Status:

Fixed

» Active

One thing I will do on that status page is make sure file_create_url is returning an absolute path.

Comment #28

Peter Bowey commented 28 May 2011 at 01:35

Status:

Active

» Fixed

Refer #26

Thanks Mike, I have zero 'hacks / mod's' in that region.

In total I only have 2 advagg mods - both to allow my 'unusual CDN URI' (1) a 'prefix' + 2) 'force a URI')

I can only 'guess' that it is the way advagg reacts to my CDN (module) mapping array.
advagg seem to just ignore it :)

No other effects...

It is not important to chase....

Comment #29

Peter Bowey commented 28 May 2011 at 01:38

Many thanks Mike!
(you are a dedicated D6/7 coder)

Comment #30

mikeytown2 commented 28 May 2011 at 04:22

Status	File	Size
new	advagg-1169702-30.patch	12.58 KB

Did it anyway; committed this patch.

Comment #31

11 June 2011 at 04:31

Status:

Fixed

» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Addressing Nginx support for advagg - mostly in 404 handling

Comments

Comment #1

Comment #2

Comment #3

Comment #4

Comment #5

Comment #6

Comment #7

Comment #8

Comment #9

Comment #10

Comment #11

Comment #12

Comment #13

Comment #14

Comment #15

Comment #16

Comment #17

Comment #18

Comment #19

Comment #20

Comment #21

Comment #22

Comment #23

Comment #24

Comment #25

Comment #26

Comment #27

Comment #28

Comment #29

Comment #30

Comment #31

News items

Our community

Documentation

Drupal code base

Governance of community