Hello,

On a Hostmonster sharing account, I am experiencing the following issue. After cron runs, the first page request by anonymous user returns the following error:

500 Server Error

A misconfiguration on the server caused a hiccup. Check the server logs, fix the problem, then try again.

On refreshing the page, the Boost-cached page loads.

Please, advise what might be the issue?

Below is my .htaccess code:

#
# Apache/PHP/Drupal settings:
#

# Protect files and directories from prying eyes.
<FilesMatch "\.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$">
  Order allow,deny
</FilesMatch>

# Don't show directory listings for URLs which map to a directory.
Options -Indexes

# Follow symbolic links in this directory.
# For security reasons, Option followsymlinks cannot be overridden.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

# Make Drupal handle any 404 errors.
ErrorDocument 404 /index.php

# Set the default handler.
DirectoryIndex index.php index.html index.htm

# Override PHP settings that cannot be changed at runtime. See
# sites/default/default.settings.php and drupal_initialize_variables() in
# includes/bootstrap.inc for settings that can be changed at runtime.

# PHP 5, Apache 1 and 2.
<IfModule mod_php5.c>
  php_flag magic_quotes_gpc                 off
  php_flag magic_quotes_sybase              off
  php_flag register_globals                 off
  php_flag session.auto_start               off
  php_value mbstring.http_input             pass
  php_value mbstring.http_output            pass
  php_flag mbstring.encoding_translation    off
</IfModule>

# Requires mod_expires to be enabled.
<IfModule mod_expires.c>
  # Enable expirations.
  ExpiresActive On

  # Cache all files for 2 weeks after access (A).
  ExpiresDefault A1209600

  <FilesMatch \.php$>
    # Do not allow PHP scripts to be cached unless they explicitly send cache
    # headers themselves. Otherwise all scripts would have to overwrite the
    # headers set by mod_expires if they want another caching behavior. This may
    # fail if an error occurs early in the bootstrap process, and it may cause
    # problems if a non-Drupal PHP file is installed in a subdirectory.
    ExpiresActive Off
  </FilesMatch>
</IfModule>

# Various rewrite rules.
<IfModule mod_rewrite.c>
  RewriteEngine on

  # Block access to "hidden" directories whose names begin with a period. This
  # includes directories used by version control systems such as Subversion or
  # Git to store control files. Files whose names begin with a period, as well
  # as the control files used by CVS, are protected by the FilesMatch directive
  # above.
  #
  # NOTE: This only works when mod_rewrite is loaded. Without mod_rewrite, it is
  # not possible to block access to entire directories from .htaccess, because
  # <DirectoryMatch> is not allowed here.
  #
  # If you do not have mod_rewrite installed, you should remove these
  # directories from your webroot or otherwise protect them from being
  # downloaded.
  RewriteRule "(^|/)\." - [F]

  # If your site can be accessed both with and without the 'www.' prefix, you
  # can use one of the following settings to redirect users to your preferred
  # URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
  #
  # To redirect all users to access the site WITH the 'www.' prefix,
  # (http://example.com/... will be redirected to http://www.example.com/...)
  # uncomment the following:
  # RewriteCond %{HTTP_HOST} !^www\. [NC]
  # RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
  #
  # To redirect all users to access the site WITHOUT the 'www.' prefix,
  # (http://www.example.com/... will be redirected to http://example.com/...)
  # uncomment the following:
  # RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
  # RewriteRule ^ http://%1%{REQUEST_URI} [L,R=301]

  # Modify the RewriteBase if you are using Drupal in a subdirectory or in a
  # VirtualDocumentRoot and the rewrite rules are not working properly.
  # For example if your site is at http://example.com/drupal uncomment and
  # modify the following line:
  # RewriteBase /drupal
  #
  # If your site is running in a VirtualDocumentRoot at http://example.com/,
  # uncomment the following line:
  # RewriteBase /

    ### BOOST START ###

  # Allow for alt paths to be set via htaccess rules; allows for cached variants (future mobile support)
  RewriteRule .* - [E=boostpath:normal]

  # Caching for anonymous users
  # Skip boost IF not get request OR uri has wrong dir OR cookie is set OR request came from this server OR https request
  RewriteCond %{REQUEST_METHOD} !^(GET|HEAD)$ [OR]
  RewriteCond %{REQUEST_URI} (^/(admin|cache|misc|modules|sites|system|openid|themes|node/add|comment/reply))|(/(edit|user|user/(login|password|register))$) [OR]
  RewriteCond %{HTTPS} on [OR]
  RewriteCond %{HTTP_COOKIE} DRUPAL_UID [OR]
  RewriteCond %{ENV:REDIRECT_STATUS} 200
  RewriteRule .* - [S=3]

  # GZIP
  RewriteCond %{HTTP:Accept-encoding} !gzip
  RewriteRule .* - [S=1]
  RewriteCond %{DOCUMENT_ROOT}/cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html -s
  RewriteRule .* cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html [L,T=text/html,E=no-gzip:1]

  # NORMAL
  RewriteCond %{DOCUMENT_ROOT}/cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html -s
  RewriteRule .* cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html [L,T=text/html]

  ### BOOST END ###

  
  # Pass all requests not referring directly to files in the filesystem to
  # index.php. Clean URLs are handled in drupal_environment_initialize().
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^ index.php [L]

  # Rules to correctly serve gzip compressed CSS and JS files.
  # Requires both mod_rewrite and mod_headers to be enabled.
  <IfModule mod_headers.c>
    # Serve gzip compressed CSS files if they exist and the client accepts gzip.
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.css $1\.css\.gz [QSA]

    # Serve gzip compressed JS files if they exist and the client accepts gzip.
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.js $1\.js\.gz [QSA]

    # Serve correct content types, and prevent mod_deflate double gzip.
    RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
    RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]

    <FilesMatch "(\.js\.gz|\.css\.gz)$">
      # Serve correct encoding type.
      Header set Content-Encoding gzip
      # Force proxies to cache gzipped & non-gzipped css/js files separately.
      Header append Vary Accept-Encoding
    </FilesMatch>
  </IfModule>
</IfModule>
CommentFileSizeAuthor
#7 boost-SymLinksOptions-2-7.x-1440754.patch2.71 KBAnonymous (not verified)
#4 boost-SymLinksOptions-7.x-1440754.patch2.6 KBAnonymous (not verified)
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

korzh-nick’s picture

I have the same error. here it is that the directory cache is written to a file .htaccess Options +FollowSymLinks , If you replace it on the Options +SymLinksIfOwnerMatch , it all works .However, when cleaning the cache file is overwritten.

//$string .= "Options +FollowSymLinks\n";
$string .= "Options +SymLinksIfOwnerMatch\n";

boost.module st.1384

bgm’s picture

Status: Active » Needs review
Anonymous’s picture

Assigned: Unassigned »

assigning myself to reviewing the clearing of the cache and possible continued removal of the .htaccess file.

Anonymous’s picture

Category: support » feature
FileSize
2.6 KB

I cannot find any example of the .htaccess file being removed during through the administration interface, it may be a cron job or crawler issue. The attached patch adds the ability to choose whether Options +SymLinksIfOwnerMatch or +FollowSymLinks is added to the cache .htaccess file.

By default the more secure SymLinksIfOwnerMatch is now selected, would stop 500 errors on hosting providers that limit the .htaccess files as mentioned above. This patch would be unlikely to cause any further issues as boost generates the cache files as the webserver user and doesn't create links (the option is only needed otherwise mod_rewrite doesn't work).

bgm’s picture

Title: 500 Server Error on the first page request after cron runs » FollowSymlinks vs SymLinksIfOwnerMatch: 500 Server Error on the first page request after cron runs

I would feel more comfortable if we defaulted on the current FollowSymlinks: Using SymlinksIfOwnerMatch has a performance cost: "Apache will have to issue extra system calls to check up on symlinks. One extra call per filename component." http://httpd.apache.org/docs/2.2/misc/perf-tuning.html#symlinks

Other comment regarding this issue (in "post-install support"):
http://drupal.org/node/1712882

The issue has also been raised for Drupal core: #1269780: Remove symlinks option from .htaccess

Also worth mentioning that mod_rewrite requires FollowSymlinks or SymlinksIfOwnerMatch to work: http://httpd.apache.org/docs/2.4/en/mod/mod_rewrite.html (I was tempted by the idea of just removing it)

Minor nitpicks on the patch:

* t('Server\'s URL or Name') : avoid escaping quotes in strings, it annoys translators. Better to use double-quotes on the string.
* #title' => t('%cache_folder Options' : Why not just t('Cache folder htaccess options') ?
* drupal coding guidelines: add a space before and after a "." (concatenation)

If you agree, can you re-roll with the default to FollowSymlinks and the above fixes?
Thanks!

Anonymous’s picture

Can we compromise ?

I will set FollowSymLinks as the default (but was trying to avoid support requests, maybe people with 500 errors will search, I'll rejig the description too), removing it is going to leave "Options None" as per the Security Advisory and mod_rewrite fails. (I have a lot of experience on other projects with that particular type of SA and cleaning out backdoors).

I will re-examine the coding standards (I did look, honest)
plus the apostrophe

t('%cache_folder Options' stays, please ?

in http://drupal.org/node/1712882 the OP quite rightly points out that the .htaccess file has to be changed in multiple places (also meaning "normal" drupal clean URL's should fail on these ISP's) and I like showing the user that there is more than one .htaccess file in play, especially as the configuration of the page is half and half, e.g. the rewrite go in root, but the headers go in the cache .htaccess.

Remember I've been approaching this with fresh eyes and going over all the older issues and as a newbie I was trying to reconcile the lack of header info in the root .htaccess, I do feel that your suggestion of a module page would help with debugging and also that some rejigging of documentation is in order, especially for the types of "my website is not caching (because they never visit anonymously)" "files expire at random (because they altered a page and a family has been expired)" "my pages never expire (because it's in the cron queue which is run weekly on their site)"

Anonymous’s picture

  • I would feel more comfortable if we defaulted on the current FollowSymlinks: DONE
  • (I was tempted by the idea of just removing it): Failed Testing: the security advisory removes all options and causes mod_rewrite to fail, would have been nice though
  • t('Server\'s URL or Name') : avoid escaping quotes in strings: DONE
  • drupal coding guidelines: add a space before and after a ".": DONE
  • Why not just t('Cache folder htaccess options') ? - Not Implemented, (see above comment about highlighting which .htaccess file is being altered)

Patch rerolled and attached.

bgm’s picture

Status: Needs review » Fixed

Committed to 7.x-1.x. Thanks for the patch!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

cssbrain’s picture

EASY SOLUTION:

sites/default/all/modules/boost

open boost.module with FTP and Notepad++

find at line 3134:
$string .= "Options +FollowSymLinks\n";

and replace with
$string .= "Options +SymLinksIfOwnerMatch\n";

and the problem is solved!

Anonymous’s picture

Apart from the fact that I patched the development version over a month ago so that it now give the user the option to use either FollowSymLinks or IfOwnerMatch and your method although it would work, would be wiped out be any updates to the module and cause 500 server errors, which is why this was marked as closed (fixed).

cssbrain’s picture

Philip_Clarke your patch is good and work!
However is difficult use patch for many people....like me.
My solution is just a little trick. Maybe useful for someone.
I was happy to share, because i spend many frustrating hours to find a solution.
Now i'm studing how to use patch. But in the meantime i need a working website.
Thank you, bye!

Anonymous’s picture

Then install the development version it should be listed in your available update and it will get you working immediately as soon as you select the ifownermatches version. It is stable except for the gzip compression which has been disbaled even if you tick the boxes.

giorgio79’s picture

Why do we need followsymlinks at all? Does Boost create any symlinks under cache?

giorgio79’s picture

Status: Closed (fixed) » Active

Reopening for evaluating followsymlinks. I doubt we need that at all under boost cache folder...

Anonymous’s picture

Status: Active » Closed (works as designed)

Closed. Mod_rewrite and boost and clean urls do not work unless the FollowSymLinks or SymLinksIfOwnerMatch options are turned on. Not having the "if owner match" option leads to 500 errors on some wen host's server configurations.

giorgio79’s picture

No need to bold the closed statement...

Boost works fine without the htaccess Options rows removed... Only thing is this seems to be some kind of security advisory related to the files directory http://drupal.org/files/sa-2006-006/advisory.txt

But that was meant for the files directory where users can upload their stuff. cache folder is maintained from a script.

Anonymous’s picture

The advisory relates to the need to stop files being executed in subfolders after a hack, and not displaying the folder contents by removing the Indexes options and limiting the folder to only presenting text or zipped files, e.g. a hacker could not place a php shell inside the folder and have it work and then adjust file permissions, exec shell scripts etc...

The main .htaccess file which contains the rewrite rules for clean urls covers symlinking but it also enables Indexes so the options directives need to remove all and the symlinks has to be put back in other mod_rewrite will not work. This is a non-issue which would result in either boost not working or the increased likelihood of file execution in the event of a hack (I spent two years cleaning servers because of this type of lack of security in an open source project), except for the 500 issue caused by server configuration on some web hosts and which has already been closed by the patch I provided.

giorgio79’s picture

Thanks Philip, appreciated for the thorough answer, your patch definitely solves the issue.

Just would like to point out that the root Drupal core htaccess has this line

# Don't show directory listings for URLs which map to a directory.
Options -Indexes

So indexes turned off everything under root, including cache dir!

If I remove options lines from boost

SetHandler Drupal_Security_Do_Not_Remove_See_SA_2006_006
Options None
Options +SymLinksifOwnerMatch

Boost and the site still works fine for me, I did test it! Also, cache folder does not show indexes. :) Just some food for thought. I still have the feeling Boost may not need those htaccess lines, if the only reason they are there is to prevent indexes. Now, those lines may solve other security issues like the ones you mentioned, but I am not sure users can upload anything there given the root htaccess is valid and alive in the cache folder also.

Anonymous’s picture

http://httpd.apache.org/docs/2.2/mod/core.html#options explains the values especially the "not normally found in the wild" mod_cgi and mod_include exploits (in my experience having removed hundreds of php shells that I won't name but wold be disabled by the current configuration) and also http://httpd.apache.org/docs/2.2/mod/core.html#sethandler explains why the files are "handled" by a non-existent handler as well as forcetype being used.

Let's say that Options is not included but -Indexes is but the default configuration is to allow server side includes where "script kiddie" has cached a file (though not through a POST request as boost would disable) where html was being parsed as a server side include and had the following in an input box or textarea.

<!--#exec cmd=".....

would then be very very dangerous. Options is only part of the belt and braces approach. Your configuration may be okay, but many web hosts are running php as part of fastCGI so have an options allowing execCGI, so without the "none" one is facing the possibility of remote script execution from a cached file.