Page not found 404 when trying to open /sitemap.xml

What is the default language, in your site?

It looks like Apache is intercepting the request for http://www.balkan-fs.com/sitemap.xml. If it were a Drupal problem you'd see Drupal 404 page. Maybe check your rewrite rules, see if there's anything that would be causing this. Also double check that in your root Drupal's directory there isn't actually a sitemap.xml file there.

For example, when we manually use the un-clean Drupal path http://www.balkan-fs.com/?q=sitemap.xml, it works just fine.

For what I can see, that is the normal set of rules Drupal comes with.

@Kiam/8: Nope, those are not the stock Drupal 6 rewrite rules.

@smoldovansky/7: Are you saying that you resolved the issue? Should I mark this as fixed?

I guess he is saying to ignore comment #7, because it was a duplicate.
There are no people who are able to write a comment 1 second after they wrote a previous one.

@Kiam, I didn't notice the timestamps, but I also can't tell when people edit comments.

I confirmed using the poster's rewrite rules will fail with loading locations like sitemap.xml and rss.xml (the default RSS feed provided by Drupal).

I'm not an expert on rewrite rules, but when I changed:
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !\.(php|ico|png|jpg|gif|css|xml|js|html?)(\W.*)?
to:
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !\.(php|ico|png|jpg|gif|css|js|html?)(\W.*)?
it worked just fine. As such, I am marking as fixed.

Looks like this has been fixed on the smoldovansky's site as well. Really marking as fixed now. :)

If the problem is the rewrite rules used, then it's not a bug of the module.
I am changing the type of the report.

I have the same symptom on a 6.x.1.1 version of the module. However when I disable the 'XML sitemap node' module the problem disappears. But then without the 'XML sitemap node' the nodes are not being listed in the sitemap.

Did you run cron after enabling the XML sitemap node?

Thanks for the reply. Yes, cron ran several times after the module was set up and no associated errors were reported in the log with the cron run.

This is a new installation of the module.
I had cleared the drupal cache as well as cleared the xml sitemap cache files using the tools.

1) What about your rewrite rules?

Clean URLs is working fine for all modules. So are URL aliases.

RewriteCond %{HTTP_HOST} ^benzinga\.com$ [NC]
RewriteRule ^(.*)$ http://www.benzinga.com/$1 [L,R=301]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

are the rewrite rules inside .htaccess

2) What does the admin/reports/dblog show for xmlsitemap entries?

Cron entries for successful submission to search engines and errors once in a while for submissions to Yahoo and Ask.

3) What is the complete list of contrib modules that you have active?

I have disabled xmlsitemap_node temporarily to prevent the 404 error.

admin_menu
adsense
adsense_managed
betterselect
better_formats
bzcustom
bzstocks
content
content_profile
content_profile_registration
ctools
date
date_api
date_timezone
feedapi
feedapi_node
filefield
globenews
googleanalytics
googlenews
imageapi
imageapi_gd
imageapi_imagemagick
imagecache
imagecache_ui
imagefield
insert_view
link
marketwire
menu_block
mollom
nodequeue
node_breadcrumb
optionwidgets
page_manager
panels
panels_export
panels_mini
panels_node
parser_common_syndication
pathauto
plus1
prnewswire
quicktabs
service_links
simplenews
simplenews_scheduler
submitagain
taxonomy_image
text
token
views
views_content
views_groupby
views_ui
votingapi
webform
webformblock
wysiwyg
xmlsitemap
xmlsitemap_engines

I don't see how xmlsitemap_node can affect the sitemap.xml URL

XML Sitemap node cannot indeed affect the sitemap URL, which is set by XML sitemap.

@anoopjohn: Do you mind if we know the link to the site you're having problems with? If you want to keep it confidential, you can use my contact form to send the link to me.

Thanks everybody for the replies. I have more information about the errors.

The 404 error that is being shown when trying to access the sitemap.xml is actually because of an internal server error that is then causing a 404 on the internal_server.html page

I checked out the apache error log and the internal server error is logged as "Premature end of script headers: index.php"

Would that be a PHP memory_limit issue? The memory limit is 90MB and it can't be increased as the site is on a Dreamhost PS which is kind of a limited VPS.

I don't have any url alias with src or dst as xmlsitemap.

I don't have a sitemap.xml in the root folder. Had there been one then .htaccess would have pointed the request to the file, woudn't it?

@earnie and @KiamLaLuno - Couldn't the xmlsitemap_node module require higher memory allocation triggering a PHP error while the module is on because of the above memory limitation and nothing when it is off?

@Dave Reid - No problem sir :-) It is www.benzinga.com.

One more piece of information. The site has 12000 nodes

I don't have a sitemap.xml in the root folder.

That is normal. XML sitemap doesn't create that file; it uses some cache files, but it will never create a sitemap.xml in Drupal root directory (also because the modules are not allowed to write into that directory).

@KiamLaLuno - Thanks for the answer. I was only answering earnie's question about whether there is a sitemap.xml file in the documentroot.

I checked out the files/xmlsitemap folder. There are no files in there. The folder has write permissions.

Drupal itself doesn't serve up internal_server.html but apache could be configured to look for internal_server.html and the actual error could be from within drupal itself. Couldn't it?

I have requested the host for the MySQL log.

The reason why I suspected that the xmlsitemap_node has something to do with this is that the moment I turn off that module, sitemap.xml becomes accessible and that the 404 error disappers. But the problem is that there would be no nodes listed in sitemap.xml.

Sorry if I was misunderstood. The internal_server.html is not shown to the end user, Drupal 404 error is shown to end user. Drupal log records that the 404 error is due to a request to an internal_server.html page.

Drupal log records that the 404 error is due to a request to an internal_server.html page.

XML sitemap doesn't try to access internal_server.html; I guess that there is something else wrong, here, and the problem is not caused by XML sitemap modules at all. Correct me if I am wrong.

I've had this happen before when my PHP memory limit ran out. PHP tried to redirect to internal_error.html which doesn't exist and Drupal serves up a 404 page. What I'm guessing might be happening is when you enable the xmlsitemap_node, it needs to regenerate the sitemap cached files with all of the node data. When it tries to do that, it runs out of memory before it can finish, so the files are not generated and you keep getting redirected to a 404. When you disable the xmlsitemap_node module, it no longer has to include your 12000 nodes in the sitemap, and so PHP doesn't run out of memory.

You might want to try out the 6.x-2.x branch which was a rewrite for performance and scalability. It's been tested on sites with thousands and thousands of nodes.

@Dave - thanks for the pointer to the 2.x branch. I have set up the 2.x branch and it seems to be working fine. I am no longer using the 1.x module so it shouldn't matter and 2.x seems to have done a lot of different things in terms of performance and I am not going back to 1.x.

I think I have found the specific part of the module that was causing the time outs and memory overshooting. It was in the xmlsitemap_node_xmlsitemap_links() function. I commented the module invoke code and the whole thing went through.

For those who come across this post - a simple message. If your site has a large number of nodes try 2.x

If your site already has a lot of nodes and you are setting up xmlsitemap module don't even think about 1.x version of the module as there is a heavy function xmlsitemap_node_xmlsitemap_links() that retrieves all the nodes in your site and processes them one by one. You will end up getting 404 errors if you enable the xmlsitemap_node module and your server has PHP memory limits or if you are using a shared host with limited processing power available for you.

2.x seems to be going beautifully well in terms of performance. So go ahead and try 2.x