The Honeytrap module allows site owners and system administrators to monitor web crawlers that do not follow the rules set out in the robot.txt file or via the RobotsText module or similar method and as a result put an unnecessarily high load on servers.
This is especially important for large, very high traffic and/or high profile sites where the activity of these non-compliant crawlers can bring servers to their knees. If these crawlers are not blocked or slowed down quickly enough then these crawlers can result in servers being knocked completely offline.
Please note that the Honeytrap module does not directly block or slow down offending IP addresses itself; it only logs and reports them, leaving you in full control of how you want to deal with them.
WARNING: When set up correctly with a firewall or similar this module will slow down and block traffic to your website. This potentially could include yourself, website owners, maintainers and legitimate visitors to your website. If you use this module then you do so at your own risk.
Requirements
Views 2 module
A configurable firewall (ideally)
A running Drupal cron task (ideally)
RobotsTxt module or access to your robots.txt file