Advertising sustains the DA. Ads are hidden for members. Join today

Contributed module documentation

Crawler Rate Limit

Crawler Rate Limit allows you to limit requests performed by web crawlers, bots, and spiders. It can also rate limit regular traffic, and block requests based on autonomous system number (ASN).

Monitor blocked requests via the command line

There is no database logging in Crawler Rate Limit, instead review rate limited and blocked requests via the web log.

Guide maintainers

ressa's picture