Hi Mikey,
Just installed the latest Dev version dated 27th September 2009.
Noticed that Boost is creating only the first directory within the cache directory and not the full path.
For example: My path is content/2009/09/27/raw-title
Boost is creating only the content directory within the cache directory.
I do not know if this is specific to my server situation and would be nice to find a solution to make Boost create the entire path with all the required subdirectories.
Thanks for all your time!

CommentFileSizeAuthor
#18 boost-590134.patch6.08 KBmikeytown2
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mikeytown2’s picture

Did Boost use to work with your path structure?

Froggie-2’s picture

Mikey, thanks for your quick response. Previously (for earlier Dev versions), I had to manually create the folder and permissions according to my path structure, so that boost is able to cache (store) the pages in their respective directories.

mikeytown2’s picture

you probably need to set the chmod settings

admin/settings/performance/boost
Boost advanced settings
Directories: Enter in a 4 digit number (octal) that will be used by chmod(). Example 0775:

Froggie-2’s picture

Thanks Mikey! I have done that already during set up where I set (octal) CHMOD() for files as 0664 and for Directories as 0775.
I have also tried by setting CHMOD for files 0777 and Directories (0777) but that did not make any difference.
Only the first directory gets created and not the full path (sub directories).

mikeytown2’s picture

To give all permissions
Files should be 0666
Directories should be 0777

That probably doesn't fix your problem though. Can I get a real URL? Use my contact form if needed.

Froggie-2’s picture

Thanks Mikey,

I have already manually created the folders (600000 +) and have uploaded them. Boost has started to cache and store the pages since yesterday but what bothers me is why this is happening. I am not too sure whether this is a situation specific to my server (VDS at GoDaddy) or this is happening with others as well who have a large amount of subdirectories in their path structure. Recently, I installed pathauto module and modified the urls of about a million nodes to avoid the apache restriction of 32k directories/files within a folder.

I shall soon email my site url to you using your contact form. I warn you though that the site is very slow at the moment as I have just started to cache the pages via Boost. Only pages cached by Boost thus far loads faster, in less than 5 seconds generally. Un-cached pages take about 20 seconds or more to get loaded due to a large amount of queries of similar nodes associated with each node/view.
Thanks again
Best Regards

mikeytown2’s picture

Dude, you weren't kidding, just shy of 5 minutes for a URL to get generated. Sounds like a MySQL problem. There is a subdir limit for boost, it's set at 10 define('BOOST_MAX_PATH_DEPTH', 10);, but yours is only 6. Once cached though, your site only takes 400ms to send the page!

System limits per directory is 32k subdirectories and about 200k files.

Here's the code in question most likely

/**
 * Recursive version of mkdir(), compatible with PHP4.
 *
 * @param $pathname
 *   The top-level directory that will be recursively created.
 * @param $recursive
 *   Operate in a recursive manner.
 */
function _boost_mkdir_p($pathname, $recursive = TRUE) {
  $mode = is_numeric(BOOST_PERMISSIONS_DIR) ? octdec(BOOST_PERMISSIONS_DIR) : 0775;
  if (is_dir($pathname)) {
    return TRUE;
  }
  if ($recursive && !_boost_mkdir_p(dirname($pathname))) {
    return FALSE;
  }
  if ($result = @mkdir($pathname, $mode)) {
    @chmod($pathname, $mode);
  }
  return $result;
}

Did you have any error show up in your watchdog that would match this format

function boost_cache_write($filename, $data = '') {
  if (!_boost_mkdir_p(dirname($filename))) {
    if (BOOST_VERBOSE >= 3) {
      watchdog('boost', 'Unable to create directory: %dir<br /> Group ID: %gid<br /> User ID: %uid<br /> Current script owner: %user<br />', array('%dir' => dirname($filename), '%gid' => getmygid(), '%uid' => getmyuid(), '%user' => get_current_user()), WATCHDOG_WARNING);
    }
  }

Or any other related errors?

Froggie-2’s picture

Thanks for visiting my site!. So far I have not seen any error or errors related to this.

A small question though, relating to system limits as stated in #7.

"System limits per directory is 32k subdirectories and about 200k files."

Question: Does it include sub sub directories? To be specific does it mean that the cache directory can hold max 32K directories, including all sub directories and all sub sub directories Or is it the case that each directory and each sub directory can each hold 32K sub directories?

Thanks again!

mikeytown2’s picture

Does this help?

Dir
     subdir
           32k subdirs
     subdir
           32k subdirs
     subdir
           32k subdirs
                        dir
                            32k subdirs
                        dir
                            32k subdirs
                        dir
                            32k subdirs
                        dir
                            32k subdirs
Froggie-2’s picture

Yep, thanks very much!

mikeytown2’s picture

Category: feature » support
mikeytown2’s picture

Component: Code » Server Hosting

I think this is specific to your hosting situation.

Froggie-2’s picture

You are right. I shall soon have to move to a dedicated server with appropriate settings. Thanks

mikeytown2’s picture

Status: Active » Postponed (maintainer needs more info)

marking this postponed, no idea how to solve this ATM

Froggie-2’s picture

Status: Postponed (maintainer needs more info) » Closed (fixed)

Hi Mikey,
I just found out by chance that disabling PHP Safe Mode on the server enables Boost to create the full path of directories and subdirectories and save the files in the appropriate directories. This means with PHP Safe Mode switched on the functionalities of Boost gets impaired on some servers. I thought I should bring this to your notice.

I am closing this issue as disabling PHP Safe Mode solved my case.

Thanks!

mikeytown2’s picture

Title: Sub-Directories not getting created for full path » Warn user on status page if running in safe mode.
Status: Closed (fixed) » Active

Thanks for the report, I will add a warning to the status page in regards to this issue.

mikeytown2’s picture

Component: Server Hosting » Caching logic
Category: support » feature
mikeytown2’s picture

Category: feature » task
Status: Active » Needs review
FileSize
6.08 KB
mikeytown2’s picture

Status: Needs review » Fixed

committed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.