Closed (fixed)
Project:
RobotsTxt
Version:
6.x-1.x-dev
Component:
Miscellaneous
Priority:
Normal
Category:
Support request
Assigned:
Unassigned
Reporter:
Created:
19 May 2010 at 09:05 UTC
Updated:
28 Jun 2010 at 22:50 UTC
Hello,
In my recent log entries I see that a Guest user hits the below link very very often (maybe a crawler?) which is causing an error: http://www.example.com/fr/tracker?component=All&category=All&priority=2&...
And the message shown is: Illegal choice 75 in Items per page element.
I have English as www.example.com and French as www.example.com/fr
The Robot.txt has the below..but do I need to add "Disallow: fr/tracker/" as well?
Disallow: /tracker/
Can this module help me?
Thanks!
Comments
Comment #1
hass commentedI think it must be "/fr/tracker/".