After updating from 6.x-1.0-rc1 to 6.x-1.0-rc2, I get this error during the CRON run:
Error occurred submitting sitemap to Google: 400.
If I go into /admin/settings/xmlsitemap/tools and run it manually, it works fine. It's only during the CRON run that it now gets this error and it didn't have this issue prior to the install of the new version. I performed the install as instructed by disabling and then deleting the old version before installing the new version.
I run the Drupal CRON job using the /path/to/drupla/scripts/drupal.sh --root /path/to/public_html/ http://default/cron.php, and all other tasks of the job appear to be working correctly. Also, I have it configured to submit to Bing as well, and that seems to work fine.
Any idea what might be causing this?
| Comment | File | Size | Author |
|---|---|---|---|
| #16 | 537140-engines-debug-D6-1.patch | 1.44 KB | dave reid |
Comments
Comment #1
stplanken commentedI am seeing the exact same thing, only with Google, only after the update to 6.x-1.0-rc2.
Comment #2
lifepillar commentedsubscribing.
Comment #3
tetsuo2501 commentedsubscribing
Comment #4
Anonymous (not verified) commentedI will be checking this tomorrow.
Comment #5
Anonymous (not verified) commentedWell, google has no problem with accessing my sitemap.xml. I'm trying the submission method for all engines now. I will check the watchdog log tomorrow.
Comment #6
buglen commentedThanks Ernie. And just to clarify, there doesn't seem to be an issue with Google requesting the sitemap or with manually submitting it from the admin page either. It's only when it tries to submit during the CRON job that it gets the error.
Comment #7
Anonymous (not verified) commentedI'm not having an issue with submitting the sitemap to any of the search engines. Though there is a bug with the frequency calculations. Did you happen to change the submission URL from the default? I don't know why you should be allowed to change it really.
Location and Hostname changed to protect my site.
Comment #8
buglen commentedHi Ernie,
I may have altered the settings with the RC1 installation, but I didn't change them after installing the RC2 version. Here's my settings for reference:
Add all the links to the default language sitemap: (not checked)
Cron limit: 100 rows
Cache directory: sites/all/files/xmlsitemap
Use stylesheet: (not checked)
Front page change frequency: Daily
Front page priority: 1.0
Promotion priority adjustment: 0.3
Comment ratio priority adjustment: 0.2
Also, the verification link for Google is not set as I'm using the meta tag in the page header via the Site Verification module (6.x-1.0-beta2).
* Edit *
Are you using the secure CRON job settings as described here (http://drupal.org/cron) in the 'Security Notes' section, or are you using the standard method with your actual domain in the CRON execution? I'm using the secure method, so maybe that could be causing the issue with the new release?
Comment #9
avpadernoCould not be that the error is caused by Site Verification, in some way?
@earnie: Do you use Site Verification too?
Comment #10
Anonymous (not verified) commentedNo, I do not use the secure method; I will have to review what exactly happens. Yes, this is most likely the issue but I do not believe any changes from rc1 to rc2 could have caused an issue as a result of this.
No, I do not use the Site Verification module yet. I plan to eventually. How would this be a cause of this issue? How would it affect a ping to the search engine URL? If this were a problem then @buglen should be able to tell from the google webmaster tools.
I'm not having an issue with submitting to the Google URL. I will give the drupal.sh method a look to see if that provides me an issue with Google. If that works, then maybe I'll add the verification module just for grins. If not, I'll debug the drupal.sh file. Else, I'll debug the verification module.
Comment #11
Anonymous (not verified) commentedHow is the drupal.sh method supposed to work with virtual multiple sites? I don't think I can use this as it is written so someone else will have to debug it. I do not use the sites/default directory and each site has its own sites/[site]/settings.php file in the same sites directory. I will now try the verification module to see if it matters.
Comment #12
buglen commentedWhile it's always worth a try, I very much doubt the site verification module has anything to do with the error. I had that module installed prior to the RC2 and it didn't cause any issues with RC1. Also, I fail to see how it could affect submission to google when you don't have any direct dependencies on that module.
If you would like me to help debug, just send me the debug enabled code to install and I'll send you the results of the extended output. Anything else I can do to help, please feel free to ask.
Comment #13
Anonymous (not verified) commentedAll I can say is to be sure to try the -dev version. Be sure to execute update.php after installing the -dev. As for a "debug enabled version" you have as much of a debug enabled version as exists. This is a minor issue for the module, though it may be critical for you. This issue is not a release blocker.
Comment #14
Anonymous (not verified) commentedDoes this happen with the 6.x-1.x-dev version? I have no issues submitting the sitemap.
Comment #15
dave reidI think in order to figure this out, we need to know the exact URL string that is being sent to google. I have a feeling that something might be getting messed up with creating the ping URL when using drupal.sh.
Comment #16
dave reidSomething like this would help us figure out what the outgoing request was.
Comment #17
Anonymous (not verified) commentedThanks, Dave. @buglen, can you apply the patch and give us feedback please?
Comment #18
buglen commented@earnie,
I would be happy to apply the patch if you could provide some instructions (or a link) on how to do so. I don't have a local install of Drupal and the module, and I don't have SSH access to my hosting account to run commands. If it can be done via a cron job to run the command, I can probably make that happen.
Just let me know.
Comment #19
Anonymous (not verified) commented@buglen: Do you have ftp access? If so you could copy the patched code to the server. The how to apply patches link is http://drupal.org/patch/apply. Ask your hosting company for help otherwise.
Comment #20
balint256 commentedPerhaps the constructed Google ping URL that xmlsitemap uses during a cron update does not have the correct sitemap URL (i.e. it constructs it using the wrong information).
E.g. if one manually pings the correct sitemap URL in a web browser:
http://www.google.com/webmasters/tools/ping?sitemap=http://my.site.com/d...
Then the request is successfully processed.
Now, if you try:
http://www.google.com/webmasters/tools/ping?sitemap=http://default/drupa...
Then Google's response is '400 Bad Request'.
I am not familiar with the drupal.sh code (specifically in relation to it using 'http://default/cron.php'), but couldn't that be the problem?
I previously used the wget method, withOUT the FQDN to my webserver (i.e. only 'www', which obviously is not reachable externally on the Internet) and the 400 error would appear in my log. It persisted with the drupal.sh method.
Now I use the 'lynx --source' method (from http://drupal.org/cron) with the externally reachable URL (i.e. just my domain name, I'm not a fan of the 'www.' prefix - try http://nasa.gov/ for kicks) and I just noticed: "Sitemap successfully submitted to Google." before "Cron run completed." in my log.
Not sure about other people's setups, but since I am within a LAN, local machines resolve my domain to my router, not my web server, so I use a proxy config PAC file to send requests with my domain as the host directly to my web server, which does vhost-ing. Therefore to make lynx connect to my web server, and not my router, I moved the 'lynx --source' line out of the cron file into a separate script. In the script the call to lynx is preceeded by "http_proxy=http://www/ProxyConfig.pac" so lynx will know exactly where to connect (the web server address used to set 'http_proxy' has to be the locally accessible version and the URL given to lynx has to be the externally accessible version!).
Hope that helps...
Comment #21
Anonymous (not verified) commentedYes, as I have already said somewhere the http://default/sitemap.xml is a direct result of the way that the drupal.sh is being used to prevent manual execution of cron.php and the $base_url not being correct because default is being used as the base url. This isn't an xmlsitemap bug but a process error in way that drupal.sh is used to execute cron.php. The assumptions being made by the documented piece about using drupal.sh for a more secure server is that sites/default/settings.php exists for the server so that it can play tricks to find the settings file.
If you set $base_url in the settings.php file does the issue go away? I cannot test this because none of my services use sites/default.
Comment #22
buglen commentedSorry, I couldn't get the patch installed for testing on my server.
However, I did see the 1.0 and 1.1 releases and read the issues for 1.0 (http://drupal.org/node/558358), which sound like they could be related. I've done a full removal of RC2 and installed 1.1, reconfigured, and did a successful manual submit to both Bing and Google. I'll let you know if it still has issues with the CRON run this morning.
Comment #24
dave reid@buglen: It was probably because of the search engine submit URLs that were messed up and were saved as empty strings instead of the proper submission URLs.
@earnie: I think we should still commit #16 so that we can debug issues like this in the future. Marking as needs review.
Comment #25
buglen commentedJust as a follow up:
Installing the 1.1 final version didn't fix the issue. However, when I assigned my domain name to the $base_url variable in the settings.php file as Earnie had suggested, the issue was corrected.
So, this isn't really a fix, but it's a work-around that is suitable for anyone else who might be experiencing this issue currently.
Comment #26
avpadernoI guess the support request can be marked as fixed, then. Feel free to re-open it if you think it is not correct.
Comment #27
Anonymous (not verified) commentedAs far as I am concerned setting $base_url for this issue is "by design". Take it up with core if you want to resolve the issue using drupal.sh such as passing an argument that gives the location of the settings.php file or setting the $base_url variable.
Comment #28
Anonymous (not verified) commentedAnd there is a patch at #16 that needs to be looked into.
Comment #29
buglen commented@earnie - I agree that setting the $base_url is a good practice to help ensure issues like this do not occur, but it doesn't explain why it worked fine in RC1 and broke in RC2 with no other changes to the site. This was the only reason I stated it was a work-around solution and not a fix, and I apologize if I sounded ungrateful.
As for the patch, I would have been happy to have applied it and return the results to you, but I am on a shared hosting solution and could not find a way to apply the patch without server access using SSH. When I get some time, I'll see if I can find a way to apply the patch to the code base locally and upload the patched code for testing.
Again, thanks for your help.
Comment #30
Anonymous (not verified) commentedI'm going to guess and say that the fix for #510076: The sitemap is not being submitted is why you began seeing these since the sitemap.xml wasn't being submitted properly before the fix. My AFAIC comment wasn't totally directed at you; it was stated as a documenting fact.
Comment #31
dave reidCommitted the patch in #16 to 6.x-1.x CVS.
http://drupal.org/cvs?commit=259506
Comment #32
stplanken commentedSetting $base_url seems to have done the trick for me. Thanks.
Comment #33
wxman commentedSame here. I was getting the 400 error too until I added the base URL
Comment #35
drupalina commentedI'm going to re-open this.
I have the exact same error showing up in my logs, but I'm using the latest stable 6x_1.1 version
please provide an outline of what is the solution to this problem? (it's not clear from the comments)
Comment #36
dave reid@drupalina: The 6.x-1.1 version doesn't have the fix that was committed. Please try the latest version and re-open if you can re-confirm.
Comment #37
drupalina commented@Dave Reid: Hi, I've disabled, uninstalled and deleted the stable 6x-1.1 version and installed the latest 6x-1.x-dev
Then I waited for my next cron to run (which I've set up with my host)
And so I still get this message. "Error occurred submitting sitemap to Google: 400"
I also got the following details:
Type xmlsitemap
Date Saturday, January 23, 2010 - 05:30
User Anonymous
Location http:///home/example/www/www/cron.php?/home/example/www/www/cron.php
Referrer
Message Error occurred submitting sitemap to Google: 400.
Request: http://www.google.com/webmasters/tools/ping?sitemap=http:///home/example...
Severity error
Any ideas?
Comment #38
dave reidSee the "My site's URLs are generated incorrectly in the sitemap." section on http://drupal.org/node/482550. You can tell because the URLs use 'http://home/' instead of a proper domain name.
Comment #39
dave reidComment #41
siliconmind commentedYesterday I've installed v6.x-1.2 - so the latest stable one.
Since then I'm experiencing the "Error occurred submitting sitemap to Google: 400".
I'm not using drupal.sh to call cron. This issue occurs even when xmlsitemap is set to send notifications on content change. So this is not an issue described above. For me it is now not clear why this happens and how to fix that. It seems that $base_url is not a workaround for me as I do not use drupal.sh.
I also have no idea why setting $base_url would help in my case - all urls from sitemap are generated correctly and also link logged with watchdog IS correct. It seems that xmlsitemap uses different url for talking to google and different one is logged with watchdog upon failure.
Dates (module release date and post #39 date) suggest that this issue should be fixed in v6.x-1.2 but is not.
It seems that this is not a minor issue as submitting the sitemap is one of the core features of this module.
Comment #42
dave reidIt still could be a problem even though you're not using drupal.sh. Can you paste the admin/reports/dblog messages about XML sitemap submission here?
Also if you had read the actual issue you'd see nothing had to be 'fixed' in the module.
Comment #43
siliconmind commentedWhen you paste this link to your browser - everything works fine and google returns 200.
Comment #44
siliconmind commentedThis is strange. Today I've noticed duplicate log entries. These four entries were created by xmlsitemap module during ONE cron run:
You said that:
So you want me to open a new one? This still relates to the 400 error.
Comment #45
Anonymous (not verified) commentedHow did you install? Was it an update from a previous module? If so did you read the instructions in the text files in the module download?
As you can tell from my questions this is appearing to me as if you upgraded without removing the previous version. See
http://drupalcode.org/viewvc/drupal/contributions/modules/xmlsitemap/INS... for more information.
Comment #46
siliconmind commentedI was updating 6.x-1.x-dev to 6.x-1.2 - this "combination" is not listed in INSTALL.TXT
Comment #47
Anonymous (not verified) commentedSorry about that, I see your point. It would depend on the age of *your* 6.x-1.x-dev install as to which path to follow. I'm going to suggest that you deactivate the xmlsitemap modules and then Uninstall them in the Uninstall tab. Then at the modules directory rm -rf xmlsitemap and then unpack the desired tar file (e.g. tar zxf /path/to/xmlsitemap-6.x-1.2.tar.gz). Reactivate the desired modules. This process does not retain any content specific settings that you may have done so you would need to redo those.
I'll also suggest you give the 6.x-2.0-alpha2 version a try.
Comment #49
srobert72 commentedToday with last DEV release :
Comment #50
srobert72 commentedChanged issue to correct version
Is URL of sitemap file correctly URL encoded as Google request it ?
Comment #51
Anonymous (not verified) commentedOpen a new issue instead of opening an already closed issue.