Hi there,

I run the latest Drupal 7 using Perusio's Nginx configuration. I used to use Filecache a long time ago on Drupal 6 with Perusio + Nginx and the cache_backport module. Now I'm evaluating it again for my site now that I'm on Drupal 7. But something isn't working, and I suspect it has something to do with the fact that I'm using Nginx.

I tried the default config for non-Apache sites:

$conf['cache_backends'] = array('sites/all/modules/filecache/filecache.inc');
$conf['cache_default_class'] = 'DrupalFileCache';
$conf['filecache_directory'] = '/tmp/drupal/filecache';

But this doesn't work at all. It still complains on the status page: "No File Cache setup in" my settings.php.

So I tried changing the filecache_directory to '/srv/www/filecache' (my Drupal is in '/srv/www/drupal'). This doesn't get rid of the "No File Cache setup in..." message. It also doesn't immediately populate the specified the changed filecache_directory until I uninstall and re-install the filecache module. And when it does finally populate it, it only generates about 66 files and doesn't seem to update them. These are the directories created inside '/srv/www/filecache' :

sites_mysite_cache         
sites_mysite_cache_field  
sites_mysite_filecache_bins
sites_mysite_cache_bootstrap  
sites_mysite_cache_rules

So for some reason it's not caching 'cache_page' or 'cache_content', which is the main reason I want to use filecache.

Any idea what's going wrong? Thanks.

Comments

sb56637 created an issue. See original summary.

ogi’s picture

ogi CreditAttribution: ogi commented

These problems suggest that there is something not right in settings.php. Could you post the result of grep cache_ settings.php? Can you also confirm that filecache is installed in sites/all/modules and so the reference to sites/all/modules/filecache/filecache.inc is correct?

ogi’s picture

ogi CreditAttribution: ogi commented
Related issues: +#1975682: Why "No cache bins are served by File Cache"? I´ve followed all readme instructions
rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Hi ogi, thanks a lot for taking the time to respond.

grep cache_ /srv/www/drupal7/sites/mysite.com/settings.php 
# $conf['block_cache_bypass_node_grants'] = TRUE;
$conf['cache_backends'] = array('sites/all/modules/filecache/filecache.inc');
$conf['cache_default_class'] = 'DrupalFileCache';
$conf['filecache_directory'] = '/tmp/drupal/filecache';

ls -l /srv/www/drupal7/sites/all/modules/filecache
total 60
-rw-r--r-- 1 nginx nginx 17519 May 21 09:14 filecache.inc
-rw-r--r-- 1 nginx nginx   307 May 21 09:38 filecache.info
-rw-r--r-- 1 nginx nginx  5293 May 21 09:14 filecache.module
-rw-r--r-- 1 nginx nginx 18092 Nov 16  2016 LICENSE.txt
-rw-r--r-- 1 nginx nginx  7314 May 21 09:14 README.txt

ls -l /tmp/drupal/filecache
total 12
drwx------ 2 nginx nginx  189 May 22 10:30 sites_mysite.com_cache
drwx------ 2 nginx nginx   84 May 22 10:30 sites_mysite.com_cache_bootstrap
drwx------ 2 nginx nginx 4096 May 22 10:30 sites_mysite.com_cache_field
drwx------ 2 nginx nginx 4096 May 22 10:30 sites_mysite.com_cache_rules
-rw-r--r-- 1 nginx nginx   87 May 22 10:30 sites_mysite.com_filecache_bins

find /tmp/drupal/filecache/ -type f | wc -l
66

I also read somewhere that the issue could be caused by some other configuration in settings.php that rewrites the $conf array. I checked for that, and only found these lines:

$conf['404_fast_paths_exclude'] = '/\/(?:styles)|(?:system\/files)\//';
$conf['404_fast_paths'] = '/\.(?:txt|png|gif|jpe?g|css|js|ico|swf|flv|cgi|bat|pl|dll|exe|asp)$/i';
$conf['404_fast_html'] = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL "@path" was not found on this server.</p></body></html>';

I don't think that should affect anything, and at any rate I tried placing the filecache conf before the 404 $conf, and it made no difference.

I also tried the previous 7.x-1.0-beta4 version, and it had the same problem. (The only difference was that there were something like 64 files all in the same level of the cache directory, per the previous design of the module.)

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Hmm, this is interesting. After a few minutes some additional bins appeared:

ls -l /tmp/drupal/filecache/
total 16
drwx------ 2 nginx nginx 4096 May 22 11:00 sites_mysite.com_cache
drwx------ 2 nginx nginx   67 May 22 11:00 sites_mysite.com_cache_bootstrap
drwx------ 2 nginx nginx 4096 May 22 11:00 sites_mysite.com_cache_field
drwx------ 2 nginx nginx 4096 May 22 10:30 sites_mysite.com_cache_rules
drwx------ 2 root  root    39 May 22 11:00 sites_mysite.com_cache_token
drwx------ 2 root  root    56 May 22 11:00 sites_mysite.com_cache_views
-rw-r--r-- 1 nginx nginx  129 May 22 11:00 sites_mysite.com_filecache_bins

Notice that everything is being created as root:root (initially the /tmp/drupal/filecache/ directory got created as root:root and I recursively chown'ed it to nginx:nginx .

Both the nginx worker processes and the php-fpm threads run as user 'nginx'. Only the master nginx controller process runs as root, and I believe it has to be that way:

ps aux | grep nginx
root      1524  0.0  0.1 135112  4980 ?        Ss   May18   0:00 nginx: master process /usr/sbin/nginx -g daemon off;
nginx     1530  0.9  0.2 135532  8316 ?        S    May18  50:39 nginx: worker process
nginx     1531  0.0  0.1 135116  6352 ?        S    May18   2:39 nginx: cache manager process
nginx     1811 14.3  1.3 373784 56580 ?        S    11:02   0:16 php-fpm: pool mysite
nginx     1812  9.5  1.4 377280 59884 ?        S    11:02   0:10 php-fpm: pool mysite
nginx     5159  9.3  1.0 368288 44300 ?        S    11:04   0:01 php-fpm: pool mysite
nginx    11848  9.0  1.2 368824 50496 ?        S    10:51   1:11 php-fpm: pool mysite
nginx    21668  5.1  1.2 371788 51912 ?        S    10:55   0:26 php-fpm: pool mysite
nginx    29277  6.9  1.4 376508 58540 ?        S    10:59   0:18 php-fpm: pool mysite

So I suspect it might actually be working, at least for those cache bins. But why is the Status page still complaining? And why won't it create the cache_page or cache_content bins? I even tried explicitly adding them to settings.php:

$conf['cache_backends'] = array('sites/all/modules/filecache/filecache.inc');
$conf['cache_default_class'] = 'DrupalFileCache';
$conf['cache_class_cache_page'] = 'DrupalFileCache';
$conf['cache_class_cache_content'] = 'DrupalFileCache';
$conf['filecache_directory'] = '/tmp/drupal/filecache';
ogi’s picture

ogi CreditAttribution: ogi commented

How do you run cron? Is it run as root by PHP CLI? Is /tmp/drupal/filecache chown nginx:nginx?

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

I use drush with Elysia cron

drush --root=/srv/www/drupal7/ --uri=http://mysite.com elysia-cron run

I was actually just about to post an issue with cron. When I try to manually run filecache_cron from the Elysia interface it gives out an ugly error page:

Fatal error: Call to undefined function filecache_registry_pathname() in /srv/www/drupal7/sites/all/modules/filecache/filecache.module on line 23

Regarding permissions:

ls -l /tmp/drupal/
total 0
drwx------ 8 nginx nginx 329 May 22 11:00 filecache
rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Could it have something to do with pathnames relative to the Drupal root in $conf['cache_backends'] , whereas it's an absolute filesystem pathname in $conf['filecache_directory'] ?

ogi’s picture

ogi CreditAttribution: ogi commented

If you have have POSIX ACL activated (probably yes), please run setfacl -R -m d:u:nginx:rwx -m u:nginx:rwx /tmp/drupal/filecache. Then please replace filecache module directory with https://www.drupal.org/project/filecache/releases/7.x-1.x-dev . Hopefully this wil fix all your issues :-)

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Thanks a lot for the fast response! I made both of those changes, but I'm not having any luck. The setfacl command didn't give any errors, but I wonder if my filesystem isn't mounted with ACL support? It's XFS, so I'm not sure how to check.

EDIT: Appears to always be enabled by default for XFS:
http://oss.sgi.com/archives/xfs/2009-12/msg00020.html

getfacl /tmp/drupal/filecache
getfacl: Removing leading '/' from absolute path names
# file: tmp/drupal/filecache
# owner: nginx
# group: nginx
user::rwx
user:nginx:rwx
group::---
mask::rwx
other::---
default:user::rwx
default:user:nginx:rwx
default:group::---
default:mask::rwx
default:other::---

And what about the parent /tmp/drupal dir? Does it need any special permissions?

ogi’s picture

ogi CreditAttribution: ogi commented

I will add checks in the status page and I will write here when it's ready for testing. Does the status page work now after updating to latest dev version?

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

No, it says:

No File Cache setup in sites/mysite.com/settings.php.
Please follow instructions in /srv/www/drupal7/sites/all/modules/filecache/README.txt.

And any idea why the cron fails with an error, and why the cache_class_cache_content and cache_class_cache_page bins aren't getting handled by Filecache?

Thanks again for your time.

ogi’s picture

ogi CreditAttribution: ogi commented

It looks like permission problem because cron is run as root and files and directories are created with owner root and cannot be modified by nginx user. setfacl should fix this and I'm surprised that it doesn't. Anyway, the status page should check for these permission problems and this is what I'm going to do.

Please confirm that the filecache_registry_pathname cron error is fixed in dev version.

I guessed that /tmp/drupal/cache is owned by root and nginx filecache cannot create the cache_page directory but it doesn't seem the case. Just in case, could you check if anonymous page caching is activated in /admin/config/development/performance and lifetimes and expire are set?

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Hi again,

The cron error continues the same:

Fatal error: Call to undefined function filecache_registry_pathname() in /srv/www/drupal7/sites/all/modules/filecache/filecache.module on line 23

Anonymous page caching is activated, but there is no lifetime or expiration configured. In my experience with other caching mechanisms such as Redis and Memcache the cache_class_cache_content and cache_class_cache_page bins still get created with these same settings.

ogi’s picture

ogi CreditAttribution: ogi commented

It seems that sites/all/modules/filecache is not the dev version from https://www.drupal.org/project/filecache/releases/7.x-1.x-dev . In the dev version, line 23 is empty line.

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Huh, you're right. That's really weird. I don't know where Elysia is getting that from. Just to be sure, I deleted and re-installed the filecache directory from the dev version you specified. Then I un-installed the filecache module and removed its config from the database. At that point Elysia cron had no filecache entry. Then I re-installed filecache and tried to run the cron entry, and it still shows me the same line 23 error. I confirm that it is indeed an empty line:

sed -n '23p' /srv/www/drupal7/sites/all/modules/filecache/filecache.module
ogi’s picture

ogi CreditAttribution: ogi commented

Could you do one more test about cache_page? Just to be sure, as anonymous user (e.g. private browser mode) go to some pages that should be cached. Then look if sites_mysite.com_cache_page is created and if there are the cached pages. If not, look at sites_mysite.com_filecache_bins - it should contain all cache bins that filecache handles.

I will make changes to the documentation about the permission problems. The correct solution is to use sudo -u nginx drush ... for all cache- and cron-related commands.

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

No, they're definitely not getting created. This is on a high traffic site, so constant anonymous hits.

This is the content of 

sites_mysite.com_filecache_bin</code?>:

<code>
a:6:{s:15:"cache_bootstrap";N;s:11:"cache_rules";N;s:5:"cache";N;s:11:"cache_field";N;s:11:"cache_views";N;s:11:"cache_token";N;}
rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

OK, partially figured out it out. Looks like my PHP opcache had the old version of the filecache module. After rebooting the entire server I remembered about that. Really sorry about the red herring / wild good chase. Anyway, the status page now works:

All cache bins except cache_ctools_css are served by File Cache.
File Cache directory /tmp/drupal/filecache uses 13.13 MB (9 min 19 sec ago)

And the filecache cron job now works as well.

However, it still isn't creating cache_page or cache_content bins.

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

After running the cron job again the Status page now shows:

File Cache directory /tmp/drupal/filecache uses 102.57 MB

But Linux says otherwise:

du -sh /tmp/drupal/filecache
832K	/tmp/drupal/filecache
rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

OK, I think I tentatively have it figured out: I moved the filecache_directory to /srv/www/filecache, which is owned nginx:nginx and has 700 perms.
Now, it shows all these bins, which continue to grow in size:

drwx------ 2 nginx nginx   4096 May 22 14:49 sites_mysite.com_cache
drwx------ 2 nginx nginx  36864 May 22 14:50 sites_mysite.com_cache_block
drwx------ 2 nginx nginx    129 May 22 14:50 sites_mysite.com_cache_bootstrap
drwx------ 2 nginx nginx 290816 May 22 14:50 sites_mysite.com_cache_field
drwx------ 2 nginx nginx  36864 May 22 14:50 sites_mysite.com_cache_filter
drwx------ 2 nginx nginx   4096 May 22 14:45 sites_mysite.com_cache_form
drwx------ 2 nginx nginx 102400 May 22 14:50 sites_mysite.com_cache_menu
drwx------ 2 nginx nginx  24576 May 22 14:50 sites_mysite.com_cache_page
drwx------ 2 nginx nginx   8192 May 22 14:50 sites_mysite.com_cache_path
drwx------ 2 nginx nginx   4096 May 22 14:40 sites_mysite.com_cache_rules
drwx------ 2 nginx nginx     39 May 22 14:41 sites_mysite.com_cache_token
drwx------ 2 nginx nginx   4096 May 22 14:47 sites_mysite.com_cache_views
-rw-r--r-- 1 nginx nginx    253 May 22 14:41 sites_mysite.com_filecache_bins

Both the Status page and du -sh /srv/www/filecache report about the same size on disk after running the cron job, and it quickly grows into the hundreds of MB. Look OK?

  • ogi committed fa30a8b on 7.x-1.x 
    Issue #2880213: Reported cache size should be the size after removing...
ogi’s picture

ogi CreditAttribution: ogi commented

It looks like the issue is solved but please don't close it. I'll make various changes related to it (permission checks in status page and updated README.txt).

I found references about cache_content in Drupal 6 modules but not in Drupal 7 modules, so it looks like there is no such cache bin in D7.

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Cool, thanks for updating this module, it's much appreciated.

It looks like filecache/sites_mysite.com_cache_bootstrap/hook_info is occasionally getting re-owned to root:root, probably after the cron job runs. If I relax the permissions to /srv/www/filecache would you see any problems with security? I'm the only user on this box, and the nginx server doesn't allow direct access to the filecache dir.

ogi’s picture

ogi CreditAttribution: ogi commented

Also, the expire of cache_page is CACHE_TEMPORARY and if cache_lifetime is 0, it always gets deleted in cron runs by filecache. Consider changing cache_lifetime in /admin/config/development/performance it to some time value of your choice. It is something that needs mentioning in README.txt

ogi’s picture

ogi CreditAttribution: ogi commented

The setfacl solution from above should work, in theory, especially now that filecache directory is not in /tmp and will not be wiped out in the next restart.

You can change FILECACHE_DIRECTORY_MODE and FILECACHE_FILE_MODE in filecache.inc to 0777 and 0666. Of course, it will get overriden by next update, so you have to watch out :-)

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Ah, that's good to know about the cache_page lifetime. But if a node gets updated or a new comment is added, will it invalidate the cache? Or would anonymous users have to wait until the cache lifetime is passed before seeing updates?

ogi’s picture

ogi CreditAttribution: ogi commented

The CACHE_TEMPORARY logic from above is specific for File Cache.

Now I see that cache_clear_all should clear cache_page and this needs to be added to filecache because it's called in many places, e.g. saving comment.

ogi’s picture

ogi CreditAttribution: ogi commented

Fortunately, no need to add it to File Cache. Core's cache_clear_all is what is called everywhere and after cleaning cache_page it calls the specific cache implementation :-)

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

So the specific permission error I'm getting is:

Warning: fopen(/srv/www/filecache/sites_mysite.com_cache_bootstrap/hook_info): failed to open stream: Permission denied in DrupalFileCache->set() (line 335 of /srv/www/drupal7/sites/all/modules/filecache/filecache.inc).

I did run your suggestion of:

setfacl -R -m d:u:nginx:rwx -m u:nginx:rwx /srv/www/filecache

Any other ideas? Thanks a ton.

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

Now I see that cache_clear_all should clear cache_page and this needs to be added to filecache because it's called in many places, e.g. saving comment.

Ah, I see. But that would clear the entire cache_page bin? Or just the entry for that page? Since my site is a high write-traffic forum it wouldn't do much good if my cache is wiped every 30 seconds when somebody posts a comment.

ogi’s picture

ogi CreditAttribution: ogi commented

I was not entirely correct. cache_clear_all calls cache_page's clear(NULL, FALSE) and this deletes only "expired" pages. If cache lifetime is 0 (none), cached pages will be deleted immediately. If cache lifetime is non-zero, this is the duration for the cached pages to be allowed to be staled. The logic in File Cache is the same as in DrupalDatabaseCache::clear

ogi’s picture

ogi CreditAttribution: ogi commented

Since your site is forum, cached pages may get deleted too often. If cache lifetime is non-zero, you risk confusing users by showing them cached pages without their just-written comment.

rahim123’s picture

rahim123 CreditAttribution: rahim123 commented

If cache lifetime is non-zero, you risk confusing users by showing them cached pages without their just-written comment.

Right, but that would only happen if they post and then immediately log out, correct? Because only anonymous users will hit the cached pages?

ogi’s picture

ogi CreditAttribution: ogi commented

(It's becoming like brainstorming but) it looks like it's OK for non-zero cache lifetime because it's used only for anonymous users, the comment author will immediately see his/her comment, just the anonymous users may see it delayed with the specified cache lifetime delay or more (cache_clear_all, cron, or something else should clear cache_page so that anonymous users to see updated content).

ogi’s picture

ogi CreditAttribution: ogi commented

The way I read the code, the exact logic when cache lifetime is non-zero is the following:

1. comment is added
2. cache_clear_all is called
3. topic page is protected in the cache because of non-zero cache lifetime, so it's not deleted
4. for the duration of cache lifetime, cache_clear_all doesn't delete the cached topic page
5. after cache lifetime is passed, the cached page is not deleted immediately
6. any cache_clear_all (any new comment anywhere) will delete the page, so it's possible that the cached topic page will not get deleted until cron, if noone calls cache_clear_all meanwhile

  • ogi committed f7df613 on 7.x-1.x 
    Issue #2880213 by sb56637: Do not allow root PHP CLI to create files or...

  • ogi committed 3aa4ad7 on 7.x-1.x 
    Issue #2880213 by sb56637: Do not allow any cache when in root PHP CLI.

  • ogi committed 47a9303 on 7.x-1.x 
    Issue #2880213 by sb56637: Any cache set while the bin is not OK tries...
ogi’s picture

ogi CreditAttribution: ogi commented
Status: Active » Fixed

Permission checks and documentation are much simplified and improved in beta6.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.