Clearing cache and running cron didn't help. I'm still hitting following error:

Boost crawler did not get a 200 response; 401 returned instead.

Do you have any ideas?

Comments

kkkatsumoto’s picture

Anyone have any suggestion for this one?

kkkatsumoto’s picture

I just applied patch from http://drupal.org/files/issues/boost-829674.patch to extend logging and heres what i've got:

Boost crawler did not get a 200 response; 401 returned instead. Authorization Required
Clear cached data & run cron; if error priests open an issue on the Boost issue queue. Got a 401 response from the URL.

kkkatsumoto’s picture

Ok, i got it solved. I had password protection in .htaccess while site is in testing phase... great for me...

kkkatsumoto’s picture

Status: Active » Closed (fixed)
merzikain’s picture

Version: 6.x-1.18 » 6.x-1.20
Category: bug » support
Status: Closed (fixed) » Active

I'm having this problem but I cannot turn off the htpasswd protection from the directory.

We have a live and dev Drupal core setup on the server and all sites on dev remain password protected. IT would throw a fit if I turned it off for any amount of time so I need to know if there's a way to have Boost (or Drupal or the apache user) autho authenticate itself?

bgm’s picture

Title: Boost crawler did not get a 200 response; 401 returned instead. » Boost crawler: support http password protection ("did not get a 200 response; 401 returned instead.")
Category: support » feature

Can you check if drupal_http_request supports http authentication? From there, might be possible to check if the httprl module supports it.

I'd much rather see this happening in 7.x-1.x (which does not have a crawler yet, see: #1074080: Cron crawler in 7.x?).
If work was done on it for 6.x, I'd create a 6.x-2.x branch where the crawler is a sub-module and relies on httprl (which has support for 6.x and 7.x, so it would be easier to manage both branches).