don't let webcrawlers try to vote on your nodes ;) [#44566]

Hi,

I noticed today that msnbot.msn.com was indexing my site because everywhere in my logs it said "access denied" (I added access control to the simplevote module):
the webcrawlers are following the links to $yoursite/vote/..., something you most likely do not want.

To stop webcrawlers from doing this, add the following line in your robots.txt (see http://drupal.org/node/22265 for more information):

Disallow: /vote

Maybe this can be mentioned in the documentation somewhere ?
greets,
-- Steven

Comments

Comment #1

ixis.dylan CreditAttribution: ixis.dylan commented 13 January 2006 at 16:49

This won't stop crawlers that ignore robots.txt. I haven't used this module, but surely the "vote" link shouldn't be displayed if the user doesn't have permission to vote? That's how most other modules works.

Comment #2

eaton CreditAttribution: eaton commented 13 January 2006 at 16:51

Assigned:	Unassigned	» eaton
Category:	feature	» bug
Priority:	Minor	» Normal
Status:	Reviewed & tested by the community	» Needs work

Actually, it IS displayed regardless of permissions. I originally intended SimpleVote as a pure API demonstration rather than a full-featured module, but I think I need to take a quick look at it and hammer out some additional bits of functionality -- like blocking unregistered users. Heh.

I'll change the basic function to write out static versions of the rating-stars if the user is anonymous.

Comment #3

eaton CreditAttribution: eaton commented 13 January 2006 at 17:26

Status:

Needs work

» Fixed

The new version posted to CVS displays spans, rather than links, to anonymous users. The net change is about 3 lines of code in the theme function. Along the way, the CSS file was also simplified a bit -- should be a little easier to see what it's doing, and redundant definitions were removed.