Drupal 8 RC1 brought spam wave to Drupal.org, which is going for a few weeks now. This issue lists steps Drupal Association staff is taking to mitigate the current wave and improve spam fighting tools long terms so the situation does not repeat itself.
Immediate actions
- B_man has been neck deep in the queues blocking alongside our valiant volunteers
- We've increased the honeypot rate limit
- Increased Mollom protection settings to strict following #2529118: Whitelist certain urls for Mollom spam checks
- Fixed Mollom not checking issue submissions at all #2535612: Check issue submissions / edits / comments for spam using Mollom
- Added rel="nofollow" to links globally and fixed it for signatures too #1548066: Signature links don't get rel="nofollow" added in all cases
Improving spam fighting tools
1. #2591025: Replace Admin content/comments views with unified view
Created single administer content view for user profiles, so it was easier for webmasters to see all content by a spammer on a single page.
2. #2125851: upgrade d.org to Flag 7.x-3.x#2386751: Replace 'report spam' links with Flag functionality
There is now 'Report as spam' flag link available on all content types, comments and user profiles. The old link which let people create Webmasters issue was removed.
Once a node, comment or user are flagged as spam, they appear in one of the following views available to webmasters:
Nodes: https://www.drupal.org/admin/content/spam
Comments: https://www.drupal.org/admin/content/spam/comment
Users: https://www.drupal.org/admin/people/spammers
3. Follow ups:
#2599918: Confirmed users should be able to unflag content they flagged as spam
#2599694: Duplicate entries in the new spammy users view
#2599388: Remove the 'approve' comment button
#2598532: Webmasters should be able to unflag content misreported as spam
#2599714: Improve bulk operations on Administer content views for user profiles
#2282473: Log bulk delete operations
#2599724: When user's content is bulk deleted and reported to Mollom, user account should be blocked automatically
#2599728: Honeypot should check more user profile fields
#2599794: New spam views need to display a message on no results
4. (Future plans) consider automatically unpublishing content after it was reported as spam X times.
Ideas/Discussion
- We're considering browser fingerprinting techniques to identify mechanical turk human spammers - but this is a complex field - still trying to find a good solution (that protects privacy).
Comment | File | Size | Author |
---|---|---|---|
#35 | spamcleanup.jpg | 170.23 KB | babipanghang |
Comments
Comment #2
B_manHello,
Drupal.org has been under a very heavy spam attack for several days now, we have mollom turned up to strict on forum posts and I have been manually removing them as fast as I can. At some points they are being created literally faster than the UI is letting me delete them. Thank you for making reports, every report helps.
Comment #3
MichelleYears ago, I asked for an admin comment view of the forum to make it easier to spot and remove spam. Did that ever get implemented? I'd be happy to help with removal but I can only do it in short bits here and there when I need a brain break so it would be nice to be able to just pop on, scan for spam, clean up, and pop off again.
Comment #4
B_manWe are working on an overhaul of the spam reporting system that will make it much easier for webmasters to manage the spam all in 1 spot.
Comment #5
MichelleThat would be great. I found my old issue... Was closed because it never went anywhere. :( #834574: Improve Adminstrative Views for Spam Fighters
Comment #6
babipanghang CreditAttribution: babipanghang as a volunteer commentedInterresting thing these spambots are even successfully registering user accounts.
They should and probably could be stopped right there in their tracks i.m.h.o.
Why is there no (re)captcha or other (apparent) security measure on the registration page?
Comment #7
B_manComment #8
MichelleIf only it were that easy, B_man. ;)
Comment #9
hestenetComment #10
hestenetComment #11
hestenetComment #12
B_manWishful thinking is fun Michelle! :D
Comment #13
tvn CreditAttribution: tvn at Drupal Association commentedThe new report as spam functionality based on Flag has be deployed yesterday (#2386751: Replace 'report spam' links with Flag functionality).
There is now 'Report as spam' flag link available on all content types, comments and user profiles. The old link which let people create Webmasters issue was removed. So we should see less (hopefully none) spam report issues in the queue.
Once a node, comment or user are flagged as spam, they appear in one of the following views available to webmasters:
Nodes: https://www.drupal.org/admin/content/spam
Comments: https://www.drupal.org/admin/content/spam/comment
Users: https://www.drupal.org/admin/people/spammers
Since there will be no more issues and hence email notifications, if you are one of the webmasters who is helping with spam fight, please check those views every once in a while.
Content and comment spam views do not have bulk operation on top intentionally. The workflow we are trying to establish is: See spam in the view, Block user account right there, click Administer content - see all content from the user on a single page - Use bulk operations there to report content to Mollom and delete it.
The spam users view only shows user accounts who were directly reported as spammers, this should help with profile spam when users do not create spam content.
There are a few follow ups we are working on:
- Let webmasters unflag something as not spam and remove it from spam views completely. At the moment this is not available. So if there was false spam report, content will stick in the view for now. We are working on fixing this asap.
- Fix bulk operations on Administer content view on user profiles so that is would report content to Mollom while deleting it.
- When webmaster is using bulk operation to delete user's content and report it to Mollom - block user account automatically.
- Remove unneeded 'approve' link from comments since it duplicated 'publish' functionality.
- Implement logging for bulk deletions, so if something was deleted by accident, we could find out what happened later.
- We'll be also looking at Honeypot config to see if we can make it stronger, and we'll make sure it checks more fields during user registration to stop more spammers from registering accounts in the first place.
We also created additional view of users to provide an easier way to search by organization and job title to identify spammers: https://www.drupal.org/admin/people/organization
Comment #14
tvn CreditAttribution: tvn at Drupal Association commentedComment #15
tvn CreditAttribution: tvn at Drupal Association commentedComment #16
tvn CreditAttribution: tvn at Drupal Association commentedComment #17
tvn CreditAttribution: tvn at Drupal Association commentedComment #18
tvn CreditAttribution: tvn at Drupal Association commentedComment #19
tvn CreditAttribution: tvn at Drupal Association commentedComment #20
tvn CreditAttribution: tvn at Drupal Association commentedComment #21
tvn CreditAttribution: tvn at Drupal Association commentedUpdated issue summary with all the follow ups.
Comment #22
MichellePlease check the access on https://www.drupal.org/admin/people/spammers . I'm getting access denied and I'm a site maintainer.
Comment #23
tvn CreditAttribution: tvn at Drupal Association commentedWill check. I am pretty sure that view exposes users' email addresses and so is only available to admins. So that was my mistake saying it is available for webmasters.
Comment #24
MichelleAh, ok. Yeah, I lost access to people's emails with the role reorganization so that makes sense.
Comment #25
tvn CreditAttribution: tvn at Drupal Association commentedComment #26
VM CreditAttribution: VM commentedJust a heads up that the spam flag cannot be reversed when erroneously clicked.https://www.drupal.org/node/2599720 is an example of an erroneously flag set by me.Thank you. Michelle.
Comment #27
MichelleThere is #2598532: Webmasters should be able to unflag content misreported as spam but that indicates you can unflag your own flaggings. If that can't be done, maybe you should note that there?
Comment #28
tvn CreditAttribution: tvn at Drupal Association commentedThanks, VM. Confirmed and going to open an issue for this. We'll be looking into it.
Comment #29
tvn CreditAttribution: tvn at Drupal Association commentedOpened https://www.drupal.org/node/2599918.
Comment #30
dddave CreditAttribution: dddave commentedWe should probably see who flagged to identify people missing the flag, shouldn't we?
Comment #31
tvn CreditAttribution: tvn at Drupal Association commentedCan you clarify what you mean dddave? Missing the flag?
Comment #32
dddave CreditAttribution: dddave commentedDamn. Autocorrect. "Misusing" was intended.
Comment #33
tvn CreditAttribution: tvn at Drupal Association commentedAh. I would suggest for now we wait and see if that happens. We have quite a few other follow ups to deal with.
Comment #34
dddave CreditAttribution: dddave commentedJust put the broom to the content and comment spam view and a bulk delete option would be handy. There were 20+ items and almost all were one-offs. This needed a lot of clicking to delete it all.
Comment #35
babipanghang CreditAttribution: babipanghang as a volunteer commentedChecking the "General discussion" forum, there are at least 5 topics that i marked as spam 2 days ago. Why are they not getting cleaned up?
I added a screenshot for reference.
Comment #36
dddave CreditAttribution: dddave commentedSigh, I am on it. The reason marked spam lingers is because there has been a "mysterious" drain of volunteer webmasters.
Comment #37
MichelleI've been trying to help but it's easy to miss stuff when the spam lists are full of bogus reports that can't be undone. :(
Comment #38
babipanghang CreditAttribution: babipanghang as a volunteer commentedI wouldn't mind helping to clean up every now and then, when i happen to be online. That is on condition that i can do or not do this whenever i choose to. If i happen to be offline for a few weeks and nobody else cleans up, i will not be accountible.
Comment #39
MichelleI've gone back through deleting spam forum nodes (not spam comments on legit nodes) thru the end of October. I didn't keep count but I'd say I must have banned 150+ spammers. There's still a lot more to be done to clean up the mess but at least the recent ones should be spam free (outside of any I missed). I'll try to work on it more as I have time but I've spent over 3 hours today and need to get my work hours in so need to stop.
Comment #40
dddave CreditAttribution: dddave commented@Michelle Can you confirm that most of those spammers are one-timers? When I check the spam nodes queue 99% are one time offenders.
Comment #41
dddave CreditAttribution: dddave commentedAll webmasters following this issue: Please go to https://www.drupal.org/admin/content and set the "published" filter to "no" and check for falsely unpublished content. I just did it after a long while and it is heartbreaking to see all the legit forum topics by newbies or requests for confirmed role screaming into a void.
Comment #42
MichelleYes, they were. I thought it was very odd because it used to be someone would make a spam account and flood with spam comments. Now it's almost always 1 account and 1 spam node. PITA to clean up because there's no mass delete for that. Have to go into every single one and delete the node and ban the spammer.
As to the unpublished, I wasn't even aware that was a problem. Is that a Mollom side effect? I'll add that to my list of things to keep an eye on.
Comment #43
dddave CreditAttribution: dddave commentedCool thanks. It was a daily habit for me but somehow I forgot. Don't bother with the spam in this queue though (or simply bulk delete it). I've already created an issue asking what is up with the unpublished releases btw.
Comment #44
MichelleIf you just bulk delete the spam, then that's not banning the spammer. Even if they typically are only spamming once, do we really want to leave all those accounts unbanned?
Comment #45
dddave CreditAttribution: dddave commentedIn this queue are a lot of accounts that have multiple spam posts. A lot of them seem to be blocked. I usually grab one or two with a lot of content when I visit the queue and block them (if necessary) before deleting the content. In a perfect world we would block each account before deleting the content but I certainly don't have the time for this (unless somebody is willing to pay me). It is more or less status quo that this unpublished spam remains untouched (for now).
Comment #46
MichelleI see. That makes sense. I've been blocking all the spammers I clean up but, yeah, it does take a _lot_ of time. I ended up taking half a day of PTO yesterday because I spent too much time on spam and didn't get my work done. Can't make a habit of that.
Comment #47
babipanghang CreditAttribution: babipanghang as a volunteer commentedAgain, i don't mind doing some cleaning up every now and then.
However, someone would have to give me the nessecary permissions to do so i guess. So what do i have to do to get these permissions?
Comment #48
MichelleYou'd need to file an issue for it. I believe this queue, user account component is where it would go. Not 100% sure but that's what makes the most sense to me.
Comment #49
mlhess CreditAttribution: mlhess commentedIf we were just to delete the spam accounts, rather then block, it we could delete the content at the same time. If they are really spam accounts......
Comment #50
hestenetComment #51
gisleNo activity for three years, and Mollom is no more. Closing as outdated.