Problem/Motivation

I am totally new to AutoBan. Just installed. My understanding is that the message patterns for rules should be the same as used for SQL LIKE or REGEXP patterns. This as per: https://drupal.stackexchange.com/a/315486

Steps to reproduce

Based upon that understanding, I am attempting to block requests to:

/topics{whatever}
/{word}.{number}.{number}

Proposed resolution

I have set my query mode to REGEXP
autoban query mode

And entered these rules:
autoban rules

Remaining tasks

Is this correct?

Also, I know the banned IPs will show up in Advanced IP log. But, is there an AutoBan log which will show which IPs were automatically banned?

CommentFileSizeAuthor
autoban rules 2024-02-13_23-07-32.jpg38.68 KBsomebodysysop
autoban query mode 2024-02-13_23-09-04.jpg28.82 KBsomebodysysop

Comments

SomebodySysop created an issue. See original summary.

somebodysysop’s picture

somebodysysop commented

So, in further testing, I discovered that AutoBan REGEXP does not accept regex start end end boundries:

It appears that this is the way to handle the word crawls (no beginning and ending word boundries):

[A-Za-z]+\.([0-9]{1,2})\.([0-9]{1,2})

But, need to know how to match in REGEXP if message starts with a particular pattern, like:

topics

goodboy’s picture

goodboy commented

Hi, Autoban uses SQL Regexp syntax, see https://dev.mysql.com/doc/refman/8.0/en/regexp.html#operator_regexp.
I've added a debug mode to the latest developer version. You can see the text of the query via the Test link, for example.

The query mode has 2 options: LIKE/REGEXP
Also, you can use Wildcards or no.

somebodysysop’s picture

somebodysysop commented

Thank you. I am using AutoBan REGEXP mode. I am also testing with "Test" option.

According to this page: https://dev.mysql.com/doc/refman/8.0/en/regexp.html#operator_regexp

To match patterns at the start of log entries or any string in SQL using REGEXP, the caret (^) symbol is indeed used to anchor the pattern to the beginning of the string.

What I am saying is that when I enter

^topics

As my pattern, the test fails.

When I enter

topics

The test returns all the log entries that begin with /topics

The problem with using the latter, is that, technically, it will match something like "?topics=", which I DO NOT want matched.

Do you see the issue? I don't think AutoBan REGEXP mode recognizes the caret (^) symbol.

somebodysysop’s picture

somebodysysop commented

Update:

I did figure out the workaround:

/topics

Will get topics the beginning of the message. Of course, theoretically it will also get:

/page/topics

But for now, this works for me.

Please let me know if I am correct about the caret (^) issue.

goodboy’s picture

goodboy commented

I did the following:

  1. Updated the taxonomy Tags and received a message in DB log "Updated vocabulary Tags."
  2. Went to the Autoban Log Analyze (/admin/config/people/autoban/analyze) and I see an entry with columns values:
    Type = 'taxonomy', Message raw = 'Updated vocabulary %name.' and Variables raw = 'a:1:{s:5:"%name";s:4:"Tags";}'
  3. Created an Autoban rule with parameters:
    Type = 'taxonomy', Message pattern = '^Updated', Threshold = 1
  4. The query mode was set as REGEXP.

I ran SQL query as
"SELECT "log"."hostname" AS "hostname", COUNT(log.hostname) AS "hcount" FROM {watchdog} "log" WHERE ("log"."type" = :db_condition_placeholder_0) AND (("log"."message" REGEXP :db_condition_placeholder_1) OR ("log"."variables" REGEXP :db_condition_placeholder_2)) GROUP BY "log"."hostname" HAVING (COUNT(log.hostname) >= :cnt)" and got IP addresses as result.

Pure SQL query is
SELECT log.hostname AS hostname, COUNT(log.hostname) AS hcount FROM watchdog log WHERE (log.type = 'taxonomy') AND ((log.message REGEXP '^Updated') OR (log.variables REGEXP '^Updated')) GROUP BY log.hostname HAVING (COUNT(log.hostname) >= 1)'and I got the same result using mysql console.

I did the same thing with the name.$ expression

So SQL REGEXP works and it's all in the expression itself. Perhaps the problem is in the escaping of special characters.

jwilson3’s picture

jwilson3
Primary language English
Location Ecuador
commented

I think I see your issue.

The Autoban module is more for scanning Watchdog log messages for behaviors that seem like repeatable patterns.

The message column for "page not found" errors in watchdog logs don't actually contain URLs like /topic. They contain a PHP "placeholder" string like @uri. Additionally, the variables column contains a PHP serialized array like a:1:{s:4:"@uri";s:9:"/topic";}. Autoban RegEx and LIKE queries both run against the raw database column contents (before placeholders are filled), therefore it will never match ^/topic.

One workaround would be to use the colon and double quote characters used in the serialized string to indicate the first part of the URL. For example: :"/topic as a "LIKE" search will turn up a match against the variables database column. The same string might also work for the RegEx-style search, but needs testing, because I don't know how well the non-ascii characters will be escaped for the regex.

On the other hand, if you want simple regex style banning from any request to specific URLs that doesnt involve scanning watchdog logs, try the Perimeter Defense module, which does exactly this.

goodboy’s picture

goodboy commented

The watchdog table has 2 columns: untranslated text pattern (on English) and the serialized variables. The human-reading text is generating by t(pattern, variables) due to current language. We can use only the unified message pattern or the variables string with serialized format.

I don't see a simple solution yet.