Problem/Motivation

Currently "paths to ignore" are processed after the bot rate limiting has been completed. This means that some of the requests that are ignored (not counted towards the rate limit) for regular traffic are not ignored for bot traffic. I can't think of any good reason for such inconsistency.

Proposed resolution

Process the "paths to ignore" before bot rate limiting is performed.

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Issue fork crawler_rate_limit-3528632

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

vaish created an issue. See original summary.

vaish opened merge request !9

  • vaish committed aa5bb216 on 3.x 
    Issue #3528632 by vaish: Process "paths to ignore" before bot rate...
vaish’s picture

vaish commented
Assigned: vaish » Unassigned
Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.