Fuss about boost_stats.php line in robots.txt [#653098]

Currently boost.install always checks for the boost_stats.php line in robots.txt

However this is only relevant is the user actually copied that file to the drupal root folder.
Therefor a check if that file exists would make sense in my opinion.

Additionally I would like to suggest reading the robots.txt not from the current directory but from $_SERVER['DOCUMENT_ROOT'] . 'robots.txt'. As a robots.txt is only requested from the domain root, and never from a sub folder.

Comment	File	Size	Author
#2	boost-653098.patch	1.2 KB	mikeytown2
#2

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

mikeytown2 CreditAttribution: mikeytown2 commented 7 December 2009 at 17:04

No, it is relevant because Boost has a menu hook called boost_stats.php. I do agree though that its kinda annoying, but I had one report of the stats block getting picked up by in google index.

Being smarter about where I get robots.txt is a good idea; I should probably download it via the url to do this check correctly.

Comment #2

mikeytown2 CreditAttribution: mikeytown2 commented 8 December 2009 at 07:28

Status:

Active

» Needs review

File	Size
boost-653098.patch	1.2 KB

Comment #3

helmo CreditAttribution: helmo commented 8 December 2009 at 07:52

Status:

Needs review

» Active

OK, I'll keep adding it to my robot.txt files.

Downloading from the site url might indeed be the safest option.

Comment #4

mikeytown2 CreditAttribution: mikeytown2 commented 10 December 2009 at 09:24

Status:

Active

» Needs review

Comment #5

helmo CreditAttribution: helmo commented 10 December 2009 at 13:53

Comments #2 and #3 must have passed each other somehow since I don't remember seeing your patch or changing back the status.

I applied the patch on my dev server and it works like I would expect.

Thanks

Comment #6

mikeytown2 CreditAttribution: mikeytown2 commented 11 December 2009 at 04:53

Status:

Needs review

» Fixed

committed

Comment #7

dbeall CreditAttribution: dbeall commented 15 December 2009 at 00:49

Just passin by with my big mouth..
"I do agree though that its kinda annoying, .... "(mikeytown2)
well, maybe.
However, Boost is such a great module and works better and better with each meticulous release, I would be more than willing to modify any file necessary to make the package work in an optimum fashion.
I just added Boost to a multi-site and it works like a charm. ! Life is good, Boost is great.
I had to do two quick little file modifications to make it work properly, htaccess and robots.txt
@mikeytown2, you just keep after every little detail, we wish other module maintainers would follow your fine example.
nice job, Thanks again