Hi - I actually got this going and it does in fact work. Both Authenticated and unauthenticated user blocks are being cached and the page is being built quite nicely.

My question: The benchmarks are quite counter-intuitive.

The results below are with 25 samples of a content-type page using jmeter. They're part of a process of before and after login looking at the same page. I did a drush cc all and a varnishadm -S /etc/varnish/secret -T url.purge . between runs (25 login sequences in each run)
Page caching = normal
Page compression = disabled
Block cache = disabled
CSS/JS aggregation = on

User type With varnish and ESI Varnish only No varnish
Authenticated 1433 368 369
Unauthenticated 68 150 50

So basically, ESI seems to make the page about 1/4 as fast. What's up with that?

Has anyone else done benchmarking of this?

The poor results with ESI turned on are the same as my experience using a browser. The blocks fill in quickly and then the content comes, and it generally feels slower than the normal.

I would appreciate suggestions of what I could do better in the configuration or benchmarking process. I've attached my vcl files.

I am admittedly a newbie to both varnish and ESI, so I take these results with a grain of salt.

esi_default.vcl_.gz456 bytesrfay
esi_blocks.vcl_.gz975 bytesrfay
Members fund testing for the Drupal project. Drupal Association Learn more


askibinski’s picture

I don't have any benchmarks right now available but I sometimes also notice some sluggish behaviour with authenticated users (more sluggish than without varnish at all). For example when logging in (or posting data) on sites which also run through varnish but aren't cashed (because they are regular D6 sites and not pressflow or D7).

The weird thing is this seems to happen mostly on chrome. Checked it on variety of computers, places and browsers. I don't think it's related with ESI because I already noticed it before using this module, but I thought I would mention it anyway.

kristofvanroy’s picture

I think the results are caused by the architecture of the module as it is right now. The creators are mentioning a second development phase addressing the root cause: "Look at the scalability issues of multiple drupal calls - possible solutions include - an esi.php back by memcache, controller esi url at the top of the page other..."

At the moment the module is using a hook_menu callback to deliver the block content. This means that you need to do a full Drupal bootstrap for every ESI on a specific page. This is actually making your site slower, because you are doing even more bootstraps.

Possible solutions:

- Serve the ESI's with a lighter bootstrap (http://api.drupal.org/api/drupal/includes--bootstrap.inc/function/drupal...)
- Capture all ESI's and fetch them all in one call

manarth’s picture

kristofvanroy is right - Unless the ESIs themselves are cached by Varnish, you will see a reduced performance.

I gave a BOF on varnish at Drupal Dev Days (Brussels) which describes some of the problems - the slides are at http://www.slideshare.net/manarth/varnish-bof (and I'll be at Drupalcon Chicago 2011 if anyone wants to talk about it).

We're working on a few key improvements:

  • Drupal 7 compatibility
  • Better out-of-the-box performance
  • Ready-to-go VCL suggestions (for both VCL versions)
  • A pluggable interface to integrate with Panels, Views, and more.

Hopefully we'll be in a position to provide more details about these at the next Drupalcon!

pounard’s picture

I also agree with #2, this is totally normal that you experience downgraded performances if your Varnish is not well configured. ESI gates are magic only if they are well used.

patcon’s picture

Re: http://drupal.org/node/1123982#comment-4335646

Any chance of in-progress D7 code being made public? :)

rfay’s picture

Would appreciate a compare and contrast with esi_api on the project page.

manarth’s picture

Short version on progress: I managed to grab a few hours with David Strauss at Drupalcon Chicago and bashed out an API plan for an extensible model for ESI in D7. I've got some basic code for 7.x-2.x - I'll move it to a sandbox tomorrow.

The esi_api module is an independent fork. It looks like one of its features is providing JS support for ESI (without using an edge-side processor such as Varnish). Pounard gives some more information on his motivation for esi_api at #1123982: Druplicate module?.

pounard’s picture

It's independant, and it's not a fork. I'm eager to see your code.

mikeytown2’s picture

Title: ESI performance does not seem approriate. Is this configuration? Anybody else do benchmarks? » ESI performance does not seem appropriate. Is this configuration? Anybody else do benchmarks?
Version: 6.x-1.0-beta1 » 6.x-2.x-dev
Status: Active » Fixed

This has been fixed in the 2.x branch. If I am mistaken please re-open.

ESI can be cached in drupal, AJAX works, and I've benchmarked it and I do see an improvement.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.