We're seeing quite a few client error and other exceptions from bots recently, it might be worth looking into blocking at least of them or specific cases.
Typical patterns:
* invalid oembed iframe requests, due to missing hash or invalid embed urls. could in theory also be valid crawlers that revisit a video that no longer exists
Symfony\Component\HttpKernel\Exception\BadRequestHttpException: This resource is not available in Drupal\media\Controller\OEmbedIframeController->render()
* invalid asset requests, such as, could also be crawlers, if the hash changes or something.
Symfony\Component\HttpKernel\Exception\BadRequestHttpException: The theme must be passed as a query argument
* attempting to send invalid headers, clearly a malicious bot
Symfony\Component\HttpKernel\Exception\BadRequestHttpException: Invalid URI: A URI must not start nor end with ASCII control characters or spaces.
* unsupported http methods, also clearly a bot
Symfony\Component\HttpKernel\Exception\MethodNotAllowedHttpException: No route found for "PROPFIND ..."
Maybe some kind of pattern that allows to match against exception class + message, or even certain log messages in general (instead of queries later on, perimeter could directly subscribe as a logger channel and immediately process log messages)
Comments
Comment #2
berdir