Hi,

I going crazy with Drupal 7 configurations and tried everything to make varnish 4 work with Drupal 7. Varnish is not working.

Below is Drupal 7 configurations:

$conf['cache_backends'][] = 'sites/all/modules/varnish/varnish.cache.inc';
$conf['cache_class_external_varnish_page'] = 'VarnishCache';
$conf['reverse_proxy'] = true;
$conf['reverse_proxy_addresses'] = array('localhost');
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 21600;
$conf['page_cache_invoke_hooks'] = false;
$conf['omit_vary_cookie'] = true;

I have taken vcl file from https://github.com/NITEMAN/varnish-bites/blob/master/varnish4/drupal-bas.... Drupal is connecting properly to varnish the issue that I am facing is that I am getting the issue below.

Cache-Control: max-age=0, no-cache

It seem like Drupal 7 is not setting up the proper cache headers. Full response is

HTTP/1.1 200 OK
Date: Mon, 11 Jul 2016 07:02:21 GMT
Server: Apache
X-Content-Type-Options: nosniff
X-Drupal-Cache: MISS
Cache-Control: max-age=0, no-cache
X-Content-Type-Options: nosniff
Content-Language: en
X-UA-Compatible: IE=edge
X-Generator: Drupal 7 (http://drupal.org)
Link: <http://stage.mysite.com/>; rel="canonical",<http://stage.site.com/>; rel="shortlink"
Vary: Accept-Encoding
X-Mod-Pagespeed: 1.9.32.14-0
Content-Type: text/html; charset=utf-8
X-Varnish-Cacheable: NO:Not Cacheable
X-Varnish: 32811
Age: 0
Via: 1.1 varnish-v4
X-Varnish-Cache: MISS
X-Varnish-Server: stage.artofmultimedia
Connection: keep-alive

Please let me know what I am doing wrong? I have tried everything Drupal is not setting cache header.

Kind regards,
Javed Gardezi

Comments

jgardezi created an issue. See original summary.

manarth’s picture

Is this for authenticated users, or anonymous?

I notice the X-Drupal-Cache: MISS header, which means that even Drupal doesn't see the content as cacheable.

For now, I would ignore Varnish, and make test requests directly to the Drupal backend, inspecting the response headers that Drupal sends. I'd then add Varnish back into the mix once I'd verified Drupal was behaving correctly.

jgardezi’s picture

This is for anonymous user.

Yes, tried without varnish and checked Drupal cache headers and getting the following headers.

Cache-Control: max-age=0, no-cache

I have also checked proper drupal performance settings. still the results were same.

manarth’s picture

The no-cache header in Drupal core is this:

Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0

As you're getting a very different cache header, and you're getting that when connecting directly to Apache instead of via Varnish, I'd suspect that there's another module injecting that no-cache header. I'd start by grepping for the literal string "no-cache" to look for another module that's adding its own headers.

jgardezi’s picture

Thank you @manarth I will have a look which module is causing it.

jgardezi’s picture

Hi @manarth

Ok I was able to fix apache server to send the following headers from Drupal

From Mac Terminal
>> curl -I http://mysite.com/

HTTP/1.1 200 OK
Date: Wed, 13 Jul 2016 07:16:13 GMT
Server: Apache
X-Content-Type-Options: nosniff
X-Drupal-Cache: HIT
Etag: "1468393490-0"
X-Content-Type-Options: nosniff
X-Frame-Options: SameOrigin
Content-Language: en
X-UA-Compatible: IE=edge
X-Generator: Drupal 7 (http://drupal.org)
Link: <http://mysite.com/>; rel="canonical",<http://mysite.com/>; rel="shortlink"
Cache-Control: public, max-age=21600
Last-Modified: Wed, 13 Jul 2016 07:04:50 GMT
Expires: Sun, 19 Nov 1978 05:00:00 GMT
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=utf-8

As you can see Cache-Control: public, max-age=21600 is sending properly. But when I use varnish I am getting the following headers.

HTTP/1.1 200 OK
Date: Wed, 13 Jul 2016 07:19:08 GMT
Server: Apache
Accept-Ranges: bytes
Vary: Accept-Encoding
X-Mod-Pagespeed: 1.9.32.14-0
Cache-Control: max-age=0, no-cache
Content-Type: text/html
X-Varnish-Cacheable: NO:Not Cacheable
X-Varnish: 23
Age: 0
Via: 1.1 varnish-v4
X-Varnish-Cache: MISS
X-Varnish-Server: varnish
Connection: keep-alive

Varnish Miss the cache X-Varnish-Cache: MISS. Please let me know what I need to change in my vcl file.

Warm regards,
Javed Gardezi

manarth’s picture

Those Drupal headers look fairly sensible, so we can assume Drupal's behaving correctly now.

There's quite a mismatch between the Apache/Drupal headers and the Varnish headers though. For example, some Drupal headers such as X-Content-Type-Options, X-UA-Compatible, and X-Generator are not being provided in the response from Varnish, but Varnish is providing headers that don't exist in the Drupal response, such as X-Mod-Pagespeed.

As there's nothing in that VCL to add or remove those specific headers, I'd guess at two possible options: either the VCL has got the backend incorrectly configured (i.e. Varnish is pointing at a different server or a different port), or something's being changed in the backend request so that Apache can't route the request to the appropriate vhost (e.g. maybe Apache is serving up the default page).

I noticed in the GitHub thread that you mentioned Varnish and Apache are running on different servers…it's worth noting that the default VCL in that GitHub repo uses a local Apache instance (127.0.0.1) by default.

So first I'd double-check the backend definition in your VCL and check it's pointing to the correct Apache server and port, and if that's correct, I'd start looking at the Apache logs to make sure the request was coming through correctly.

jgardezi’s picture

I have checked my .host configuration in varnish it was pointing to my staging IP address. I have changed varnish vcl backend .host = staging IP to the production IP address of the backend apache server.

Yes, you were right that apache was serving the default page from backend staging server.

The apache production IP is configured for multiple v-hosts. Eventually, I want all the v-hosts to be served from varnish.

So I changed my varnish vcl backend .host to production IP and I can see the website was severed correctly instead of staging apache default page.

From Mac terminal (for mysite.com)
>> curl -I http://mysite.com/ this is response I got.

HTTP/1.1 200 OK
Date: Thu, 14 Jul 2016 00:11:22 GMT
Server: Apache
X-Content-Type-Options: nosniff
X-Drupal-Cache: HIT
X-Content-Type-Options: nosniff
X-Frame-Options: SameOrigin
Content-Language: en
X-UA-Compatible: IE=edge
X-Generator: Drupal 7 (http://drupal.org)
Link: <http://mysite.com/>; rel="canonical",<http://mysite.com/>; rel="shortlink"
Cache-Control: public, max-age=21600
Last-Modified: Wed, 13 Jul 2016 22:08:59 GMT
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Vary: Accept-Encoding
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=utf-8
X-Varnish-Cacheable: YES
X-Varnish: 131084 10
Age: 715
Via: 1.1 varnish-v4
ETag: W/"1468447739-0-gzip"
X-Varnish-Cache: HIT
X-Varnish-Cache-Hits: 3
X-Varnish-Server: varnish
Connection: keep-alive

Varnish worked fine for host mysite.com. So I went and tested this (same varnish server) for another Drupal host on the same production server.

From Mac terminal (for anotherwebsite.com)
>> curl -I http://anotherwebsite.com/ this is response I got.

HTTP/1.1 200 OK
Date: Thu, 14 Jul 2016 01:22:39 GMT
Server: Apache
X-Content-Type-Options: nosniff
X-Drupal-Cache: HIT
Content-Language: en
X-Generator: Drupal 7 (http://drupal.org)
Link: <http://anotherwebsite.com/metadata_image.jpg>; rel="image_src",<http://anotherwebsite.com/>; rel="canonical",<http://anotherwebsite.com/>; rel="shortlink",<http://anotherwebsite.com>; rel="publisher"
Cache-Control: public, max-age=3600
Last-Modified: Thu, 14 Jul 2016 01:14:54 GMT
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Vary: Cookie,Accept-Encoding
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=utf-8
X-Varnish-Cacheable: YES
X-Varnish: 196613 32799
Age: 658
Via: 1.1 varnish-v4
ETag: W/"1468458894-0-gzip"
X-Varnish-Cache: HIT
X-Varnish-Cache-Hits: 9
X-Varnish-Server: varnish
Connection: keep-alive

Varnish worked fine for host anotherwebsite.com.

It seem like production IP address in varnish vcl backend .host works fine for multiple v-hosts in apache. I though about modifying varnish vcl sub vcl_recv to be like this

sub vcl_recv {
 if (req.http.host ~ "^mysite.com(:[0-9]+)?$") {
    set req.http.host = "mysite.com";
 } else if (req.http.host ~ "^anotherwebsite.com(:[0-9]+)?$") {
    set req.http.host = "anotherwebsite.com";
 }
 ....
 ....
}

but I guess at this stage modifying/configuring sub vcl_recv is not required. Can you please advise that modifying sub vcl_recv for host specific is required to improve the varnish cache HIT ratio or have any other advantages?

Warm regards,
Javed Gardezi

manarth’s picture

The most common use-case I know of for modifying the HTTP Host header in vcl_recv is normalising the domain.
For example, you may handle both example.com and www.example.com, and normalising the URL will improve the cache-hit rate.

It's not all that common to rewrite the Host header though, and in most cases it will be absolutely fine without it.

jgardezi’s picture

@manarth Thank you for your help.

manarth’s picture

Title: Drupal 7 Cache-Control max-age=0, no-cache issue » Varnish configuration and cache-control headers
Category: Bug report » Support request
Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.