Sites Robots.txt

Sites Robots.txt lets every site managed by the Sites module serve its own robots.txt. It bridges Sites with the RobotsTxt module so that the file served at /robots.txt is built from the currently active site — different domains on one Drupal install get different robots.txt content.

Features

• Per-site robots.txt content, edited directly on the site.
• Child sites inherit their parent site's robots.txt (child returns 404).
• Optional: return 404 for /robots.txt when no site context is active.

Requirements

Drupal 11+, the Sites module and the RobotsTxt module.

Getting started

Edit a site and fill in the "Site specific robots.txt additions" field. Global behaviour is configured at /admin/config/search/sites-robotstxt.