Problem/Motivation

With the increasing number of AI bots crawling sites and causing problems, both for Views with exposed forms Facets 3 filters (facets as checkboxes) and Facets 2 (facets as links) I wonder if a Honeypot Facets sub-module could be possible? Idea from #3444984-4: Add support for views-exposed-form.

See also:

Perimeter or Tarpit could be candidates for a link-driven Facets Honeypot solution, for Facets 2, where Facets are links, see #3547754: Anyone using Perimeter as a Facets 2 bad bot blocker?.

Background

AI bots are not actually submitting forms, but the bots love clicking through links and the Facets 2 issue boils down to the fact that Facet 2 blocks aren't really checkboxes, but in fact just links styled to look like checkboxes.

Facets 3 switches to using actual checkboxes and forms instead of lists of links, which is promising and will probably save us for now.

Eventually, bots will probably get more advanced and start clicking Facets 3 checkboxes. Maybe they already do?

A more custom UI just for Facets 3 blocks could be possible, but probably complex, while on the other hand, it is easily doable from a few lines of custom form_alter code.

Steps to reproduce

Get overrun by AI bots, and want to block them by offering them Honeypot Facet checkboxes (Facets 3) and links (Facets 2) , which when clicked will block the IP address.

Proposed resolution

Add support for Facets Honeypot for bots, for both links (Facets 2) and checkboxes (Facets 3).

Remaining tasks

User interface changes

API changes

Data model changes

Comments

ressa created an issue. See original summary.

ressa’s picture

Issue summary: View changes
jwilson3’s picture

Follow! My understanding is that AI bots are not actually submitting forms, but that the bots love clicking through links and the Facets issue boils down to the fact that Facet blocks aren't really checkboxes, but are in fact just links styled to look like checkboxes. Facets 3 switches to using actual checkboxes and forms instead of lists of links, which is promising and will probably save us for now.

But as time wears on and bots get more advanced, there might be real opportunity here. But my suggestion would be to first try adding additional form ids for the Facets 3 block forms to the existing Honeypot module settings on your site, to guage whether Honeypot + Facets forms it is having any effect in rebuffing bot requests. #2342473: Allow for more flexible form configuration seems like a promising solution, if you're not opposed to manually entering facet form ids. A more custom UI just for Facet blocks could be possible, but probably complex, while on the other hand, it is easily doable from a few lines of custom form_alter code.

Here is what I'm doing to add Honeypot to all Webforms. Presumably it could be adapted for Facet blocks as well assuming they follow some kind of standard form id prefix in their naming conventions.


function mymodule_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  // Add honeypot protection to all Webforms.
  // @todo remove when https://drupal.org/i/3544510 lands
  if (strpos($form_id, 'webform_submission') === 0) {
    if (\Drupal::moduleHandler()->moduleExists('honeypot')) {
      // There is no reason to not enable time restriction here since
      // Honeypot respects the time_limit set in the configuration.
      \Drupal::service('honeypot')->addFormProtection($form, $form_state, ['honeypot', 'time_restriction']);
    }
  }
ressa’s picture

Issue summary: View changes

Thanks for the encouraging comment @jwilson3!

Your description of the situation between Facets 2 and 3 is correct, and a great summary, so I added it in the Issue Summary. I also now see I use Facets 2, in the intro text, where it really should be Facets 3, so great to get that corrected as well.

Thanks for sharing your custom solution for Webform. I am in the middle of upgrading from Facets 2 to 3, and will assess the situation, and if needed try to adapt the code. If I can get it to work, I can add it to a doc page, maybe https://www.drupal.org/docs/administering-a-drupal-site/security-in-drup...?

protitude’s picture

I'm currently running facets 3 and a hook_form_alter didn't seem to work for me. I created a module that just injects a hidden checkbox at the top of my facets via javascript and if it's checked an Event Subscriber adds an entry to watchdog. That's all my module does, and from there you can create a rule in autoban to look for that entry and ban any bots checking those boxes.

For me, I add all my facet pages to my Robots.txt so any bots coming to those pages and checking everything shouldn't be there at all. So it seems safe to ban them after clicking a hidden checkbox.

You can find my module here: https://github.com/UCBoulder/facets_honeypot

Feel free to fork it or use it as is. Any suggestions to improve it is welcome, though I'm trying to keep it simple.

I'd be happy to add facets_honeypot to this module as a sub module, but I'm not sure if it's getting too far away from what the honeypot module was originally developed for, though I'm not sure this module should be a standalone module on d.o either.

ressa’s picture

Issue summary: View changes

Fantastic news, thanks for building it @protitude!

And you're probably right, that adding it as a Honeypot submodule might not be the right method, since after taking a closer look at Honeypot, it doesn't have any submodules .... Also, since your module does not rely on Honeypot, perhaps creating it as a stand-alone module on drupal.org under https://www.drupal.org/project/facets_honeypot makes most sense?

By having it as an official project on drupal.org it will get much more exposure, and more users, and it will surely become quite popular, since it solves a problem, which is only growing bigger every day, for more and more web sites. So feel free to add it here if you agree, it would be a fantastic addition! On the other hand, if you prefer to not create a new standalone project, a Honeypot submodule would also work fine.

PS: I actually tried and failed to update to Facets 3, to transition from links to checkboxes (see #3354129: Update project page with new branch details). I think many others will also stay on Facets 2 with Facets as links, until those blocking AJAX and Block placement issues are fixed. Perhaps Facets Honeypot could eventually also offer links as traps for bots? But we can always revisit that later :)

ressa’s picture

I am still very interested in a drupal.org version of the Facets Honeypot module, and maybe even a future feature, of links as Honeypots as well?

In the meantime, I just discovered the very interesting module Tarpit, which could fill the niche of a link-based Honeypot. There is a coding standards issue, and after patching, it works well.

ressa’s picture

I was working on a Perimeter README MR today (#3427559: Add Whitelisted IPs, IP-Ranges, and expanded ban pattern example in README) and realized that Perimeter might be a great candidate for a link-driven Facets 2 Honeypot solution, where Facets are links: #3547754: Anyone using Perimeter as a Facets 2 bad bot blocker?.

What do you all think about that idea?

ressa’s picture

Title: Honeypot Facets 3 submodule? » Honeypot Facets 2 and 3 submodule for links and checkboxes?