Problem/Motivation

Given the age of crawlers, bots, AI agents etc crawling sites, it's very likely for a client to receive an unwanted wave of traffic that crawls their search pages, submitting queries, selecting facets, filets, etc en masse, leading to rapid exhaustion of entitlements and unwanted cost or loss of search functionality.

The acquia_search module offers configurable flood limits to the user, see here.

```
/**
* List of values by each Solarium request type.
*/
private static function getFloodDefaults() {
return [
'select' => ['window' => 10, 'limit' => 50],
'update' => ['window' => 60, 'limit' => 600],
'update/extract' => ['window' => 60, 'limit' => 600],
'autocomplete' => ['window' => 10, 'limit' => 100],
'test' => ['window' => 2, 'limit' => 1],
];
}
```

These limits are in place with Acquia Search - they are configurable via drush - and their role is to prevent unwanted traffic traversing search pages from creating Solr queries at a rate that would quickly exhaust the client's query allowance per month. Essentially it's a stopgap, that if activated will stop the search service with an error to the user, thus alerting the site owner who can then take action blocking the unwanted traffic.

It would be nice to have these additional limits to prevent users from exhausting their available quotas too quickly due to any bad actors.

Proposed resolution

If possible, provide a way for users of the SearchStax Drupal module to place additional limits beyond the default Site Search API Gateway limits.

Issue fork searchstax-3544493

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

kaunaj created an issue. See original summary.

magtak’s picture

Following

magtak’s picture

I wrote a proof of concept submodule to demonstrate this better:

https://github.com/magtak/searchstax_flood

@drunkenmonkey can you have a look?

drunken monkey made their first commit to this issue’s fork.

  • drunken monkey committed f965dda0 on 1.x
    feat: #3544493 Added configurable rate limiting in the module.
    
    By:...
drunken monkey’s picture

Status: Active » Needs review

@magtak: Thanks for your work! I used this as the base for an MR adding this functionality, in a configurable way, to the module – and just managed to get the pipelines green on it.

I now merged the MR to HEAD and will continue testing in the dev version of the module.
There will be a new release once all new features and fixes have been confirmed.

  • drunken monkey committed 2e700823 on 1.x
    fix: #3544493 Fixed two problems with rate limiting.
    
    By: drunken monkey
    
drunken monkey’s picture

I just merged a second MR which fixes two problems with the previous implementation:

  1. Form validation of our settings form produced errors when window/limit fields were left empty.
  2. Autocomplete requests where not covered by the “search” flood protection.
drunken monkey’s picture

Status: Needs review » Fixed

This has now been confirmed to work as intended and released as part of version 1.10.0.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

magtak’s picture

Thank you @drunken monkey! :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.