I am getting a lot of page not found errors. The cached page is sending a page that calls an optimized css or js file that does not exist. If I clear cache manually it corrects the problem.

Here is the watchdog error.

Type	page not found
Date	Tuesday, November 24, 2009 - 3:09am
User	admin
Location	http://www.thetriton.com/sites/default/files/js/js_a03196619cc8835d991dfb059162b650.js
Referrer	http://74.125.23.132/search?q=cache:DcrHZxQ8yvUJ:www.thetriton.com/events%3Fmini%3Dcalendar%252F2009-11-02+usb+triton-270&cd=51&hl=en&ct=clnk&gl=us
Message	sites/default/files/js/js_a03196619cc8835d991dfb059162b650.js
Severity	warning
Hostname	71.166.112.11
Operations	

Comments

JirkaRybka’s picture

I assume that http://74.125.23.132/search?q=cache:DcrHZxQ8yvUJ:www.thetriton.com/events%3Fmini%3Dcalendar%252F2009-11-02+usb+triton-270&cd=51&hl=en&ct=clnk&gl=us%3Cbr%20/%3EMessage is the referrer? If so, then I see these as well, but it's not a fault of Drupal at all: The HTML simply got cached by a third party search engine, and is therefore re-used later, without the Drupal installation knowing about it. The referenced js file doesn't exist anymore (most probably because the Drupal cache got flushed). If the page was requested from the Drupal site properly (i.e. on the domain thetriton.com in this case), Drupal would serve the HTML _and_ rebuild proper css/js being referenced. In this case though, the HTML is served from a third-party cache at http://74.125.23.132/search?q=cache, and only the js file alone is requested from the original domain, that did exist at the time the cache was fetched, but doesn't exist anymore.

This IMO doesn't have any relation to Cache Router, or any other caching backend being used.

Adam S’s picture

It happened to me while I was logged in as Admin. I went to my events page and the calendar did not show anymore. http://thetriton.com/events . Nobody else had that problem for a couple of hours so I'm going to assume that it was also associated with an automatic cron run (four times a day). When I ran clear cache it was fine again.

I'm just going to assume that this is a robot. I found this under recent hits. It did not show under reports. I wonder why?

URL	http://thetriton.com/sites/default/files/js/js_0a999fa2c481997554abf9b78e29f91b.js
Title	Page not found
Referrer	http://www.thetriton.com/
Date	Tuesday, November 24, 2009 - 8:01pm
User	Guest
Hostname	82.128.42.64
JirkaRybka’s picture

I don't know what to say, as this doesn't say that much (there was a bug in core, as far as I remember, where visiting Themes page without submit removed the aggregated files without flushing page cache, so anonymous users temporarily got non-existent referenced file - might be this one as well, or something similar). But still, this is about Drupal, not about Cache Router. Cache Router does nothing about these files, it only just provides alternative storage engines for already existing system.

You might get more attention as Support request for Drupal project, where this IMO belongs.

crea’s picture

Category: bug » support
Status: Active » Fixed

Cache router can't interfere with these files: it's only cache. Btw unless you or your visitors see unstyled pages (hence their browsers can't find CSS files) it should be okay ( bots/searchengines can store outdated paged as said above).

jpp’s picture

There are two fixes for this.

1) find the lines of code in includes/common.inc that do a file_scan_directory() and delte the js and css - once you've found them comment them out and use a cron job to age out css and js files beyond a certain age. This is a gross kludge

2) if you are running lighttpd write some lua code to check if the requested file exists and if it doesn't find and serve the latest one instead.

That code looks like this (build on top of the clean urls drupal.lua code) - you will need to install the lua filesystem module

-- prefix without the trailing slash
local prefix = ''

require "lfs"
-- helper function to find missing css
function findLatest(path,mask)
  candidate = ""
  cmod = 0
  for file in lfs.dir(path) do
     if string.find(file,mask) then
        fattr = lfs.attributes(path .. file)
        if (candidate == "" or fattr.modification > cattr.modification) then
           candidate = file
           cattr = fattr
        end
     end
  end
  return(path .. candidate)
end

-- the magic ;)
if (not file_exists(lighty.env["physical.path"])) then
    -- see if it's a missing system generated css
    if(string.find(lighty.env["physical.path"],"css/css_[%w]+%.css")) then
      -- See if we can find the latest css file
      _,_,path = string.find(lighty.env["physical.path"],"(.+/css/)css_[%w]+%.css")
      css = findLatest( path,"css_[%w]+%.css")
      if (css) then
        lighty.env["physical.path"] = css
      end
    elseif (string.find(lighty.env["physical.path"],"js/js_[%w]+%.js")) then
      -- See if we can find the latest css file
      _,_,path = string.find(lighty.env["physical.path"],"(.+/js/)js_[%w]+%.js")
      js = findLatest( path,"js_[%w]+%.js")
      if (js) then
        lighty.env["physical.path"] = js
      end
    else
    -- file still missing. pass it to the fastcgi backend
      request_uri = removePrefix(lighty.env["uri.path"], prefix)
      if request_uri then
        lighty.env["uri.path"]          = prefix .. "/index.php"
        local uriquery = lighty.env["uri.query"] or ""
        lighty.env["uri.query"] = uriquery .. (uriquery ~= "" and "&" or "") .. "q=" .. request_uri
        lighty.env["physical.rel-path"] = lighty.env["uri.path"]
        lighty.env["request.orig-uri"]  = lighty.env["request.uri"]
        lighty.env["physical.path"]     = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"]
      end
    end
end
-- fallthrough will put it back into the lighty request loop
-- that means we get the 304 handling for free. ;)

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.