Using filecache my file cache folder is around 5 million files and 25GB in size.
The cron job always "fails" with no hint after less than 100K calls to unserialize so my cache folder never shrink. I had to add a shell cron job to delete files older than 10 days and keep some control on the size.
I saw the code has a constant define('FILECACHE_CRON_MAX_FILES', 100); but this is not used anywhere.
Also my status page keeps telling me that the directory size is unknown and I should run cron (indeed I did it).
What could be done to make it work?
I thoung about changing the folder structure to something like this:
tempfolder/substring(md5(path),0,2)/path
This way we would have 256 equally distributed folders. Each cron call could work on a single folder and then the next cron would work on the next folder (256 cron calls to run the full "gc").
This sounds like an "hack" but I didn't find anything better.
Otherwise filecache could keep an "expiration index" so to know what files need expiration without opening and unserializing them all. Maybe a single file named #expiry#-#random# and containing the name of the file to be expired. This way you could simply open files with a given name and the expiration would be much lighter, but this means dealing with twice the files (and fileops) during standard cache writes.
WDYT?
Comments
Comment #1
vinoth.3v CreditAttribution: vinoth.3v commentedAny Update?
I am getting memory limit error on Cron.
Comment #2
gunwald CreditAttribution: gunwald commentedI have the same problem: Cron always fails with the message:
While cron fails after a while I have to many files in /tmp that no more file can be created: df -i
This is a critical bug because it brings the server down not only the Drupal site.
So, if you have a site with many pages, blocks, etc. don't use this module until this bug is fixed!
Comment #3
vinoth.3v CreditAttribution: vinoth.3v commentedrelated Issue #1211166: Create separate directories for each cache bins