Hi

I really would like to use Varnish caching with the robots.txt so we can avoid Drupal bootstrap at all. My bootstrap is around 200ms and Varnish can serve the file in 75ms and no apache thread.

For this I think would be best to use standard page caching mechanism as this is easily setup globally and supported by core.

This is basically the same solution as in this issue for the Redirect module #1392974: cache control headers are not correctly added to redirects

CommentFileSizeAuthor
#7 send_cache_headers-1923838-7.patch568 bytesdokumori
#5 send_cache_headers-1923838-5.patch393 bytesrbrandon
robotstxt-cache-headers.patch542 byteskars-t

Comments

dawehner’s picture

dawehner
Primary language German
commented
Status: Active » Needs review

:)

kars-t’s picture

kars-t commented

Oh how could I forget this -_- thx!

hass’s picture

hass commented

Should we better issue drupal_page_footer() than drupal_exit()?

dawehner’s picture

dawehner
Primary language German
commented

Yeah why not!

dawehner’s picture

dawehner
Primary language German
commented
Issue summary: View changes

Enhanced text

rbrandon’s picture

rbrandon commented
Issue summary: View changes
StatusFileSize
new send_cache_headers-1923838-5.patch393 bytes

This is good enough for varnish if someone would like a simpler patch in the interm.

Why not have the function return NULL instead of drupal_exit()? This will call drupal_deliver_page as usual without sending any extra data which will call drupal_page_footer() etc.

dokumori’s picture

dokumori commented

+1 to @rbrandon's suggestion on returning null. I may be short-sighted, but if the objective here is simply to return robots.txt, I don't see why implementations of hook_exit need to be called.

I'll modify @Kars-T's patch and see how it goes.

dokumori’s picture

dokumori commented
StatusFileSize
new send_cache_headers-1923838-7.patch568 bytes

Will test and report back:

hass’s picture

hass commented

Should we go with #5 now?

hass’s picture

hass commented
Status: Needs review » Needs work

I guess we should go with drupal_page_footer() as we had the same discussion already in #1858178: call to exit in callback function breaks page caching, but it seems like we have implemented the wrong fix with drupal_exit().

hass’s picture

hass commented
Status: Needs work » Fixed

http://drupalcode.org/project/robotstxt.git/commit/cc4e8f0

  • Commit cc4e8f0 on 7.x-1.x by hass: 
    Issue #1923838: Send correct cache headers so we can cache the
robots....

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

hass’s picture

hass commented
Status: Closed (fixed) » Needs work

Suxxx. This may introduced #2296351: Exit after drupal_page_footer()

hass’s picture

hass commented
Status: Needs work » Closed (fixed)