By BramDriesen on
Change record status: 
Published (View all published change records)
Project: 
Drupal core
Introduced in branch: 
9.5.x
Introduced in version: 
9.5.0
Issue links: 
https://www.drupal.org/node/3271222
Description: 

Drupal's default robots.txt file (used for informing web crawlers what paths to not index) has been updated to disallow indexing of Oembed media links. This change only added following rules.

...
Disallow: /media/oembed
Disallow: /*/media/oembed
...
Disallow: /index.php/media/oembed
Disallow: /index.php/*/media/oembed
...

File changes:
- core/assets/scaffold/files/robots.txt
- robots.txt

Impacts: 
Site builders, administrators, editors
Updates Done (doc team, etc.)
Online documentation: 
Not done
Theming guide: 
Not done
Module developer documentation: 
Not done
Examples project: 
Not done
Coder Review: 
Not done
Coder Upgrade: 
Not done
Other: 
Other updates done