This module allows you to add additional lines to each domain's robots.txt file. The most frequently used scenario is probably to exclude a certain domain from being indexed by search engines entirely. You would use:
Disallow: /

Requirements:

The admin settings form is at:
admin/structure/domain/robotstxt

Project Information

Downloads