Hello,

This may not be detected on a basic Drupal installation, but let's first compare both behaviors in particular cases, then we'll see how this can affect cache management. Let's say that Drupal runs at root on example.com.

Calling cache_clear_all() runs cache_clear_all(NULL, 'cache_page');, which in turn calls (pseudo-code)
new MyDrupalCacheClass('cache_page')->clear(NULL, FALSE);.
MyDrupalCacheClass is usually DrupalDatabaseCache, but is of course changed to VarnishCache when using Varnish. clear() class method has two different implementations, DrupalDatabaseCache::clear() and VarnishCache::clear(). I'm focusing on their behavior when called by cache_clear_all(). Then:

  • DrupalDatabaseCache will remove all expired cache from cache_page table.
  • VarnishCache will, instead, call varnish_purge_all_pages() which will run, in turn, the Varnish command
    purge req.http.host ~ example.com && req.url ~ "/" which means, in more natural words "clear Varnish cache items which have a host that matches example.com, on all URLs" (please correct me if I'm wrong on this one)

Now this would work fine, only if our Drupal website has one and only one reaching domain name, like our example.com above. Now what happens in both those cases?

  • We have a multilingual website and a different domain for each language: example.com for English, and example.fr for French. Well, Varnish will only clear page cache on the English side.
  • We use Domain Access module, and have a lot of various domains that reach this site (that can be a subdomain, like subdomain.example.com, or totally different domains or aliases). Then Varnish will only, once more, clear a fraction of the page cache (only all cache saved under subdomain.example.com). Luckily this time, if cache_clear_all()is run on example.com, then all (but nothing else) Varnish cache saved under any-subdomain.example.com will be cleaned (still not perfect).

Would there be a way to rewrite the Varnish command call to stick to DrupalDatabaseCache::clear() behaviour?

Thanks for your help,

David

Comments

David Stosik’s picture

David Stosik commented
Issue summary: View changes

Improve readability.

DouglasZero’s picture

DouglasZero commented

I was facing the same problem.

@David Stosik, thanks to your post i have understood the problem and found a solution.

In my case, excluding the:

req.http.host ~ example.com &&

Solved the problem.

This is not a final solution, but works if you are in a dedicated server.

szds’s picture

szds commented

As ban allows regex, hence,

ban req.http.host ~ example.com$ && req.url ~ /

will ban all urls from hostnames that end with example.com
e.g.
a.example.com
b.example.com ...etc

If you follow cato's patch in https://drupal.org/node/1812944
you can set Varnish Clear Host to "example.com$"

JeremyFrench’s picture

JeremyFrench commented

This is complicated.

You are right in that varnish cache works differently to DrupalDatabaseCache::clear() however there are very different use cases. Some people would not want this to do anything. As on a high traffic website clearing the whole cache can be fatal. However for others this will be too strict.

The trend is for this module to clear less rather than all but I think that I should make it clear (in the readme) what options there are currently and then we can address any missing use cases.

bibo’s picture

bibo commented

You are right in that varnish cache works differently to DrupalDatabaseCache::clear() however there are very different use cases. Some people would not want this to do anything. As on a high traffic website clearing the whole cache can be fatal. However for others this will be too strict.

I agree, but I would add that an admin setting to either flush varnish per domain or globally, would be a simple solution. In my case I wanted to flush different domains for a few reasons, mainly because of CDN-domains.

Because there is currently no option to remove the domain-filter (like DouglasZero in #1), I created a very simple drush extension ("drush vpg"), that does the same thing, just directly instead of varnish_purge():

_varnish_terminal_run(array('$command req.url ~ "$pattern"'));

More details in the related issue, which is almost a duplicate of this one.

bibo’s picture

bibo commented
Related issues: +#2017097: Ban/purge requests to varnish are malformed.

To be more specific, I think this is the "master"-issue: #2017097: Ban/purge requests to varnish are malformed.

IMO either that or this one should be marked as a duplicate. This one is older, but the other one has actual "progress" with a patch.
What do you think @JeremyFrench?

bibo’s picture

bibo commented
Related issues: +#1323418: Support for Expire module option expire_include_base_url=1

I think I'll just mark this as duplicate for now. Please correct me if you think this isn't a duplicate.

Actually this issue seems to be about the same (base_url) problem too, and it is older than this #1323418: Support for Expire module option expire_include_base_url=1.

Nevertheless, this issue has the most progress (I guess):
#2017097: Ban/purge requests to varnish are malformed.