hi

since i installed the friendly url's, i think they have become a bit too friendly, for bots at least. i see google adding comments to my nodes.

the first time this happend i set the comment setting to "must preview", then "submit" and this solved the problem. now it seems google is using the traceback link to post an empty comment

this is see in my logging of drupal:
Type special
Date Wednesday, 27 August, 2003 - 08:24
User kritsch-AVRO-lid
Location /myblog/tb/2789?op=form
Message comment: added ''
Hostname 64.68.82.168

(note: [root@kjell root]# host 64.68.82.168
168.82.68.64.in-addr.arpa domain name pointer crawler14.googlebot.com.)

in my apache logging i trace this action from this host to:
64.68.82.168 - - [27/Aug/2003:08:24:30 +0200] "GET /myblog/tb/2789?op=form HTTP/
1.0" 200 83 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

this happends for 20 nodes per day. all comments added are empty.

has anyone else seen this? what could i change in my config to let google crawl my site (not blocking it by a robots.txt) and yet have a clean insightfull comment system?

Comments

Boris Mann _Old Blogger.com Account_’s picture

Very strange! I've got Google doing very complete crawls of my site, and I've never seen something like this.

Sorry, can't be more helpful on what to do, since I don't really understand what Google could be doing!

--
Boris Mann

bertboerland’s picture

i clicked a traceback link myself and saw i got an error and a comment in my node. disabled tb, will dig into later, most likely error on my config

--
groets

bertb

--
groets
bert boerland

Boris Mann _Old Blogger.com Account_’s picture

I actually installed and enabled tb myself and got the same error (clicked on the link and got empty comment).

I'll dig into the tb.module at some point. It should have a filter that checks for a) search engine URLs or b) your own site as referrer.

Actually, I want to see the referrer portion of statistics used *instead* of traceback -- then, you automatically see referrer links, and neither side has to do anything more complicated than link to a node/item/etc. (see the comment on my site).

--
Boris Mann