Using XML Sitemap with ModSecurity™

ModSecurity™ is an open source intrusion detection and prevention engine for Web applications (or a Web application firewall). Operating as an Apache Web server module or stand-alone, the purpose of ModSecurity™ is to increase Web application security, protecting Web applications from known and unknown attacks. While ModSecurity™ can be a very powerful tool, misconfigured or overly strict rule sets can interfere with the submission settings option of the XML Sitemap module and return an error like this:

406 error
Not Acceptable
An appropriate representation of the requested resource /admin/settings/xmlsitemap/engines could not be found on this server.

You can submit the site map by manually visiting each search engine site, but you can also temporarily disable ModSecurity™ to quickly submit your site map to all the search engines by adding the following to your .htaccess file to override the ModSecurity™ settings on Apache:

<IfModule mod_security.c>
   SecFilterEngine Off
   SecFilterScanPOST Off
</IfModule>

Upload your .htaccess file and go to http://www.example.com/?q=admin/settings/xmlsitemap. Select Submit site map on cron run (under the search engines tab) and click on Save configuration. Because you are overriding ModSecurity™, you can then go to http://www.example.com/cron.php to manually run cron. Once that’s finished, if you go to http://www.example.com/?q=admin/logs/watchdog, you should see a log entry for your search engine submissions. You can now re-enable ModSecurity™ by removing the lines you added to your .htaccess file.