Testing our antispam measure

I'm not sure how much valid testing we can do on a test also, but I agree with your philosophy erring on the side of the user is better. However, our main spam problem is not really the occasional human created spam that might also suck in legit comment. The main issue, and the reason this whole issue surfaced again for real, is mass spam from spam bots.

In any case, the first thing I encountered was the following when marking an item as spam using the spamtest account:

imo, the user marking spam should not receive this message.

Also, the fact that only spam admins can mark spam seems somewhat counter intuitive and rather than adding our own custom 'mark as spam' link, the module should be adjusted to separate out this permission from 'administer spam'. I haven't had a chance to check the spam module's issue queue yet, but maybe we can submit a patch if one hasn't been submitted already.

maybe we can test the patch in #1014114: Test the spam administration permission... could we have two levels??

another thought-- we'd want vbo actions for 'mark as spam' and 'unmark spam' so we could add them to our admin-nodes and admin-comments views (nothing shows up admin/content/content).

@4-1: weird. I keep getting it. i get http://spam-drupal.redesign.devdrupal.org/spam/denied as the url with the above image. Unmarking spam works fine. Maybe it matter which posts? Can you try http://spam-drupal.redesign.devdrupal.org/node/1412168?

@4-2: imo this is pretty important. We don't really need webmasters to have a 'mark as spam' link, but users. You're aware of the issue with using flag for this, and I'm afraid after we deployed it would go into the black hole of never getting done. I think we stand a better chance of getting users the ability to mark spam if we require this patch for deployment.

@4-3: can definitely be post-deployment.

hmmm.... good point. I guess we can always just create a 'report spam' link that creates a webmaster issue completely separate from the spam module (since using flag for this any time soon seems unlikely), but I would have thought this type of functionality would be a common feature and part of the spam module.

without this, I'm a little a on the fence about deploying the spam module at all since, although the filters and other features are nice, it doesn't do the two most important and simple things we need most as part of our initial anti-spam changes: 1) automate spam reporting for users (as opposed to site maintainers) and 2) flood control.

Obviously spam module can do those things if the module maintainer accepts the features, but do we really want to add an entire module like the spam module for 2 simple functions we're going to have to write anyway.

thoughts?

Hi guys,

A quick set of answers:

1. Permissions are lacking. Authenticated users don't get a 'Mark as spam' link unless that role has the "Administer spam" permission - but that gives them access to change the filters (which isn't cool.) If there's a way around this, I'd like to know!

I have a patch proposed for that purpose. #1014114: Test the spam administration permission... could we have two levels?

1.5 Therefore, we may need that "Report as spam" link, anyway.

See (1).

2. The Bayesian filter may add too many tables to the database - but this might be a 'wait and see' how big the table gets.

I suppose you meant "too many words to the database table". The Bayesian filter does not add new tables as it grows. Also, older words (unused for a while) get removed from the table.

3. Testing spam on a test site is somewhat futile, since it doesn't get the 50-500 spam posts d.o gets, not to mention legitimate posts.

Yeah. I agree that it is difficult to test the spam filtering mechanism...

----

Personal insight: at this point no one started work on a version for D7, is that a problem for you guys?

Thank you.
Alexis Wilke

yeah-- i commented above that i was on the fence. but I think i've come off the fence since and am now firmly in the 'we should find another way' camp.

I suppose this is the appropriate status, then.

spam.module is still my favourite choice for antispam measures. When I tested it for a client, I didn't have the problems that silverwing encountered.

I guess I need to do some testing myself with the mentioned patches applied.

I'm in the backend of http://spam-drupal.redesign.devdrupal.org/ now, per #1596432: I want access to the drupal.org development site for testing spam prevention issues (dman)

Not sure what next. Do we have to write our own spambot to hammer the thing to test?

I've been working on this. See #1293186-138: Spam - meta: better spam-combating suggestions.

I guess we can always just create a 'report spam' link that creates a webmaster issue completely separate from the spam module (since using flag for this any time soon seems unlikely), but I would have thought this type of functionality would be a common feature and part of the spam module.

I've applied the split permission patch, and developed report_spam.module on top of it (which is a bit more sane way for our users to flag content as spam) for this purpose. Based on the user role, report_spam will increment the spamminess score of a piece of content. When it passes the threshold, it gets unpublished. When it gets unpublished, it also trains the bayesian filter.

I'm getting "This post is spam" when I don't think it should be giving me that error as Administrator

This sounds like a bug, but there is a "bypass filters" permission that might need to be applied. If you're logged in as Dries, though, then I'm not sure what's going on.

editing book page nodes with a lot of links (as a lot of our pages have) also gets marked as spam.

I built a domain whitelist for the URL filter for this purpose. That way, we can specify that any link at drupal.org, devdrupal.org, or drupalcode.org is safe and we don't need to count it against the user.

without this, I'm a little a on the fence about deploying the spam module at all since, although the filters and other features are nice, it doesn't do the two most important and simple things we need most as part of our initial anti-spam changes: 1) automate spam reporting for users (as opposed to site maintainers) and 2) flood control.

Number 1 is handled by report_spam.module, and Number 2 is partially handled by the duplicate content filter. I'll do it the real way and contribute a flood control filter to spam.module, though. That should be in the next couple of days - should be pretty easy to write something that just provides a control for x posts per n minutes per user threshold.

Tagging.

deployment note

Closing, antispam measures changed somewhat since 2012. There are more recent issues.