Problem/Motivation

Thanks for creating this module, it looks really interesting! The closest module could be Crawler Rate Limit? But since it requires setting up extra backend, this module looks easier to set up, and get started faster.

Also, it seems like using this module could be a more forgiving method, than out-right blocking aggressive crawler IP's, since you might actually want to allow bots and crawlers from search engines and LLM's access, but limit them for example to one request every 5th second. If true, it looks to me like this module could be the best alternative to hard block, as it is outlined under Block bad bots and crawlers > Block bot IPs in .htaccess.

Steps to reproduce

Proposed resolution

Perhaps a short sentence could be added, clarifying these two questions on the project page and maybe README?

Also, a link to the Protect Views Flood Control module could be added in the Block bad bots and crawlers > Block bot IPs in .htaccess section, with a mention that it might serve as a viable alternative, to facilitate a more gentle crawling.

Remaining tasks

User interface changes

API changes

Data model changes

Comments

ressa created an issue. See original summary.

scott_euser’s picture

scott_euser commented

Hmmm yeah I can see small overlap, though I personally have no problem with AI going wild on cached pages (great: better training for LLM + more likely to get AI Overview traffic), really this module is specifically for views exposed form which Protect Form Flood Control doesn't cover per #3444984: Add support for views-exposed-form.

Do you have a suggestion for what text could go on the homepage? I haven't used crawler rate limit because I'd normally do that via Cloudflare

ressa’s picture

ressa
he/him
commented

Thanks for a fast answer, and just to be clear, I do think this module is a great addition - the other module works very differently.

It could be something like this?

There's a slight overlap with Crawler Rate Limit, but it requires more configuration, where this module is simpler to set up.

Some block aggressive IP's manually via .htaccess, where this module could serve as an automated alternative.

scott_euser’s picture

scott_euser commented
Status: Active » Fixed

Thanks! In the end I modified what you wrote to fit in the alternative/complementary approaches section of the module homepage: https://www.drupal.org/project/protect_views_flood_control#module-project--alternatives

Thank you

ressa’s picture

ressa
he/him
commented

That's a great method, laying out the multitude of options available to solve this challenging task, thanks!

Though I did add a caveat about blocking via .htaccess, so a link to the main doc page https://www.drupal.org/docs/administering-a-drupal-site/security-in-drup... could be considered instead? I will attempt to keep it up to date, with the latest developments in this ever evolving space. Perhaps something along these lines? (feel of course free to re-word it):

PS. I was sort of looking forward to testing if Facets 3 checkboxes might make a difference, but it looks like the bots are taking a breather the last few days, after all their hard work recently ... :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.