Problem/Motivation
Given the age of crawlers, bots, AI agents etc crawling sites, it's very likely for a client to receive an unwanted wave of traffic that crawls their search pages, submitting queries, selecting facets, filets, etc en masse, leading to rapid exhaustion of entitlements and unwanted cost or loss of search functionality.
The acquia_search module offers configurable flood limits to the user, see here.
```
/**
* List of values by each Solarium request type.
*/
private static function getFloodDefaults() {
return [
'select' => ['window' => 10, 'limit' => 50],
'update' => ['window' => 60, 'limit' => 600],
'update/extract' => ['window' => 60, 'limit' => 600],
'autocomplete' => ['window' => 10, 'limit' => 100],
'test' => ['window' => 2, 'limit' => 1],
];
}
```
These limits are in place with Acquia Search - they are configurable via drush - and their role is to prevent unwanted traffic traversing search pages from creating Solr queries at a rate that would quickly exhaust the client's query allowance per month. Essentially it's a stopgap, that if activated will stop the search service with an error to the user, thus alerting the site owner who can then take action blocking the unwanted traffic.
It would be nice to have these additional limits to prevent users from exhausting their available quotas too quickly due to any bad actors.
Proposed resolution
If possible, provide a way for users of the SearchStax Drupal module to place additional limits beyond the default Site Search API Gateway limits.
Issue fork searchstax-3544493
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
magtak commentedFollowing
Comment #3
magtak commentedI wrote a proof of concept submodule to demonstrate this better:
https://github.com/magtak/searchstax_flood
@drunkenmonkey can you have a look?
Comment #7
drunken monkey@magtak: Thanks for your work! I used this as the base for an MR adding this functionality, in a configurable way, to the module – and just managed to get the pipelines green on it.
I now merged the MR to HEAD and will continue testing in the dev version of the module.
There will be a new release once all new features and fixes have been confirmed.
Comment #10
drunken monkeyI just merged a second MR which fixes two problems with the previous implementation:
Comment #11
drunken monkeyThis has now been confirmed to work as intended and released as part of version 1.10.0.
Comment #13
magtak commentedThank you @drunken monkey! :)