This patch removes the obsolete paths /database/ and /update/ and reorders the other paths and files by alphabet.

This patch needs to be backported to D6 and D5 as both also contains the obsolete path's.

CommentFileSizeAuthor
D7_updated_robotstxt.patch1.04 KBhass

Comments

dries’s picture

dries commented
Version: 7.x-dev » 6.x-dev

I've committed this to CVS HEAD. Thanks. Lowering version to Drupal 6.

hass’s picture

hass commented
Status: Needs review » Reviewed & tested by the community

Patch also applies cleanly to D6 and D5.

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

I am a bit reluctant to commit this patch. On the plus side, it might prevent disallowing indexing of directories, the user might create in her Drupal installation, such as /database and /updates, so that might be an improvement for Drupal 6. On the minus side, it is unlikely that these paths are used and robots.txt is often edited, so we should reserve to changing it on production branches only when required, not for "cosmetic" changes. Otherwise we just give people some headache to diff and update their robots.txt for the new version without much benefit.

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

Pending on feedback from http://lists.drupal.org/pipermail/development/2008-November/031378.html

alexanderpas’s picture

alexanderpas commented
Title: Updated robotxt.txt » Updated robots.txt
Priority: Normal » Minor
gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented
Version: 6.x-dev » 5.x-dev

Ok, committed to Drupal 6 to coincide with the .htaccess change as well at: http://drupal.org/node/28776

drumm’s picture

drumm
he/him
Location NY, US
commented
Version: 5.x-dev » 6.x-dev
Status: Reviewed & tested by the community » Fixed

Committed to 5.x.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.