Problem/Motivation

While writing Drupal 12 release notes it came up that robots.txt still contains coverage for the removed search module. This should also be removed.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Robots were blocked from indexing search result pages with query strings in Drupal 11.4.0. As Search module was removed from Drupal 12, this part of robots.txt was also removed. If you are moving to the contributed Search module, you'll now need to manually add it back to your site's robots.txt. See https://www.drupal.org/node/3627873

Issue fork drupal-3627629

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

gábor hojtsy created an issue. See original summary.

gábor hojtsy opened merge request !17328

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented
Status: Active » Needs review
smustgrave’s picture

smustgrave commented
Status: Needs review » Reviewed & tested by the community

Marking this as it makes sense with search but should some CR be written? Imagine if you're switching to contrib you'll have to edit your robots now and that's not a step most may not be aware of.

xjm’s picture

xjm
she/her
Primary language English
commented
Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs change record, +Needs release note, +12.0.0 release notes, +12.0.0-beta2 release notes

CR is a fair suggestion. We should also add a sentence about this to the RN about Search's removal... and actually a separate RN specifically for beta2.

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented
Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs change record, -Needs release note

Hm yeah this was added in #3550083: robots.txt should disallow all '/search/' and '/search?' pages by default originally and released in https://www.drupal.org/project/drupal/releases/11.4.0, where the release notes have this mentioned too. I was expecting we did not actually release this yet, but we did. So indeed we need a reverse release note / change notice.

Added change record and release notes snippet.

dcam’s picture

dcam commented
Status: Needs review » Reviewed & tested by the community

The CR and release notes look good to me. I don't have any feedback for them. I'm moving this back to RTBC.