Arduino Forum

Forum 2005-2010 (read only) => Software => Bugs & Suggestions => Topic started by: follower on Aug 17, 2008, 05:35 am

Title: Spam fighting request
Post by: follower on Aug 17, 2008, 05:35 am
Hi,

I would like to repeat my request for something to be done about the spam on the forums.

Can someone please organise to allow myself or one of the other forums regulars the ability to delete spam directly--in my opinion it is inefficient to be marking it manually and I don't think it's a good use of the existing moderators' time to be searching for threads marked SPAM and deleting them.

--Phil.  
Title: Re: Spam fighting request
Post by: Digger450 on Aug 17, 2008, 07:31 am
Yes, it is getting old.
Title: Re: Spam fighting request
Post by: mellis on Aug 17, 2008, 10:07 pm
follower: it's not a bad idea.  I'll discuss it with the team.  I've also been thinking about trying to divide up the hardware interfacing forum into multiple categories, so it could be nice to have some extra moderators to help out in categorizing the posts.

OTOH, I don't think there's so much spam to really interfere with the forum usage, although I understand that it's annoying.
Title: Re: Spam fighting request
Post by: follower on Aug 20, 2008, 07:42 am
Quote
I'll discuss it with the team.

I'd appreciate it.

Quote
I've also been thinking about trying to divide up the hardware interfacing forum into multiple categories, so it could be nice to have some extra moderators to help out in categorizing the posts.

FWIW IMO I'm not sure there'd be a huge benefit from dividing up the hardware forums. People already seem to have problems knowing quite which sub-forum to post in and I don't think the volume of postings is dangerously high. What do you see as being the gain from doing so?

Quote
OTOH, I don't think there's so much spam to really interfere with the forum usage, although I understand that it's annoying.

I guess I subscribe to the "broken window" theory and (particularly after today's deluge) I don't think it's getting any less. I don't think anybody gains from leaving spam on the forums longer than is necessary.

--Phil.
Title: Re: Spam fighting request
Post by: jds on Aug 20, 2008, 09:51 am
I didn't see this thread and posted a similar thread in the News section... Sorry!
Title: Re: Spam fighting request
Post by: walterr on Aug 20, 2008, 04:56 pm
I would also like to see the spam problem improved.
Title: Re: Spam fighting request
Post by: admin on Aug 20, 2008, 05:35 pm
Quote
I would also like to see the spam problem improved.



I don't see it as a huge problem... it's not that we have a 90% spam 10% content ration..
we are cleaning the forums every day multiple times...

Title: Re: Spam fighting request
Post by: zea on Aug 21, 2008, 04:55 am
what about to use a captcha system for the borad?

http://www.yabbforum.com/community/YaBB.pl?num=1186785854/0
Title: Re: Spam fighting request
Post by: Oracle on Aug 21, 2008, 05:33 pm
Quote
what about to use a captcha system for the borad?

http://www.yabbforum.com/community/YaBB.pl?num=1186785854/0


I'm not conviced captchas work, when I added them to my own site it barely slowed them down.  Though by the time I added them, I was getting about 200 spams a day and legit traffic had pretty much died out.

From my experience though, it is important to do something more than just deleting the spam.  It started out as a trickle on my site, and grew and grew until it was almost a full time job deleting them.
Title: Re: Spam fighting request
Post by: westfw on Aug 23, 2008, 11:13 pm
Quote
I don't see it as a huge problem...

But the spam is getting more frequent AND more offensive...

Title: Re: Spam fighting request
Post by: follower on Aug 28, 2008, 05:33 pm
Hi...

Quote
it's not that we have a 90% spam 10% content ration..

That's not a very high standard to compare the situation to! :-)

To be honest I don't understand the reluctance to enable a couple of people to implement this. Is there something about the situation I'm not understanding?

A greater than zero number of community members find the spam unnecessary and there are at least a couple of people willing to provide the mouse clicks required to delete it more promptly. Less spam. More time for the existing mods to do more useful things. And I could delete the spam with one click (I assume) instead of typing in a bunch of stuff and waiting for a moderator to delete it.

I would appreciate it if you could please explain the reason for this not being an acceptable solution for you.

Thanks.

--Phil.

P.S. FWIW with the current spam volume I think captchas would inconvenience the forum community more than the spam.
Title: Re: Spam fighting request
Post by: Oracle on Aug 28, 2008, 07:37 pm
Quote

To be honest I don't understand the reluctance to enable a couple of people to implement this. Is there something about the situation I'm not understanding?


From the point of view of the core team, Arduno is their baby, the built it up, and it's very hard to let go of control.  It's the same sort of feeling that parents have as their children grow up.

As the Arduino community grows, I suspect it will become more and more necessary to have some help, but I can certainly understand the reluctances of the core team to give up control of their baby.

I'd say just be patient right now, they are dealing with the spam.  The only real downside right now is they're spending time that could go into helping users or further developing Ardunio on deleting spam.

As the legitimate user base grows, it's not simply deleting spam, but more subjective things mods would have to do.  Moving threads to more appropriate forums for example.  Should a mod only delete spam or a grossly off-topic thread?  Where do you draw the line?  
Title: Re: Spam fighting request
Post by: John_Ryan on Aug 28, 2008, 08:33 pm
Quote
Quote
what about to use a captcha system for the borad?

http://www.yabbforum.com/community/YaBB.pl?num=1186785854/0


I'm not conviced captchas work, when I added them to my own site it barely slowed them down.


The attacks Arduino.cc's has been suffering, are Bot attacks, and bots "cannot" navigate sites protected by "good" implementations of CAPTCHAs, such as the format used over at Yahoo.

The general mentality of spammers is if they can let their bot's roam the web injecting their filth into millions of vulnerable forums, like this one, then it beats doing any form of work themselves, like reading site CAPTACHA codes for a single attack, especially if the pay-off isn't high unlike the persistent attacks suffered by WindowsLive and Gmail whose CAPTCHAs were hacked, which considering the "kudos" prize, its no wonder.

So for the 5 minutes the Yabb hack takes to implement it'll reduce a substantial amount of the misery users here are obviously suffering as a result of sometimes highly offensive spam.


Title: Re: Spam fighting request
Post by: Oracle on Aug 28, 2008, 11:00 pm
Quote


The attacks Arduino.cc's has been suffering, are Bot attacks, and bots "cannot" navigate sites protected by "good" implementations of CAPTCHAs, such as the format used over at Yahoo.


It's trivial to break any captcha.  You simply link it into a "free porn" site and have users answer the captchas.  They type the captcha, the bot automatically enters it into the victim site, and if it's right, the script lets the viewer have their porn.   There is no such thing as a good implementation because any captcha can be defeated with this sort of attack.
Title: Re: Spam fighting request
Post by: John_Ryan on Aug 29, 2008, 12:34 am
Quote
Quote


The attacks Arduino.cc's has been suffering, are Bot attacks, and bots "cannot" navigate sites protected by "good" implementations of CAPTCHAs, such as the format used over at Yahoo.


It's trivial to break any captcha.  You simply link it into a "free porn" site and have users answer the captchas.  They type the captcha, the bot automatically enters it into the victim site, and if it's right, the script lets the viewer have their porn.   There is no such thing as a good implementation because any captcha can be defeated with this sort of attack.


I haven't heard of that one before, how does it work?

If CAPTCHAs are so easy to defeat, then the brains over at Carnegie Mellon University have a lot to be held accountable for, and gosh, the cheek of it, they're still telling people to use it!

http://www.captcha.net/

Our portfolio includes a number of adult sites we've owned "for years", all with CAPTCHA, not one has ever suffered a spam attack. An adult forum we own started out with Yabb, but it got too big and we upgraded it to SMF. In its 3 year history only a handful of Russians have tried manual attacks, but the site has 3 moderators and they blocked the IP's before they had a chance to do anything.

Spam is completely preventable.  
Title: Re: Spam fighting request
Post by: follower on Sep 02, 2008, 03:18 pm
Quote
From the point of view of the core team, Arduno is their baby, the built it up, and it's very hard to let go of control.

Okay, so I guess I had hoped that isn't the reason.

I know what it's like to transition a project to grow it and I know it's not easy. But I'm kinda concerned if we can't even move to enabling spam deletion. The forums are part of the community and IMHO we have all built that.

Quote
I'd say just be patient right now, they are dealing with the spam.

From my point of view it's not the first time the issue been raised and I don't think the spam is being dealt with--there's been certain posts that have stayed around for 2-3 days without being deleted--despite being marked as SPAM.

Quote
The only real downside right now is they're spending time that could go into helping users or further developing Ardunio on deleting spam.

While that's one downside it's not the only one--it takes more time to mark something as SPAM than it does to delete it.

Quote
Should a mod only delete spam or a grossly off-topic thread?  Where do you draw the line?  

As far as I'm concerned I'm only interested in deleting spam. Grossly off-topic threads can already be dealt with by community nudges in the right direction.

I'm not wanting to come across as a jerk but I would like to know if we're likely to see any change in this because IMHO it's an unnecessary negative in the community.

--Phil.
Title: Re: Spam fighting request
Post by: BigMike on Sep 09, 2008, 12:50 pm
I also find the spam objectionable.

There are technical solutions and volunteers - why not use them?

Mike
Title: Re: Spam fighting request
Post by: Digger450 on Sep 09, 2008, 07:43 pm
It really does seem to be getting worse, why not address it now?  I've skipped wading through the messages several times in the last week due to the spam content.
Title: Re: Spam fighting request
Post by: follower on Sep 10, 2008, 02:59 pm
Okay, this is getting ridiculous, there was so much spam today that when I was marking it I got given this message!

Quote
Preview - The last POST request from your IP was less than 5 seconds ago, and was blocked to prevent overloading the forum. Please try later.

This would all be fractionally less irritating if it was possible for a normal user to change the subject used in the thread listing but AFAICT the only way to make it obvious to others that a thread is spam would be to change to using a differently named user account.

What do we have to do for this to be taken seriously?

--Phil.
Title: Re: Spam fighting request
Post by: mem on Sep 10, 2008, 03:23 pm
Spam is pushing real posts off the 'recent' list  making it inconvenient to find new posts. Can the 'action= recent' count be increased to some number that allows viewing of posts over at least the last hour and ideally the last 12  hours or so.
Title: Re: Spam fighting request
Post by: Oracle on Sep 10, 2008, 03:49 pm
This is getting really bad, we're past the point of it "not being a huge problem"  and might be getting to that "90% spam 10% content ratio" that Massimo talked about.

FWIW, I'm willing to help any way I can.
Title: Re: Spam fighting request
Post by: follower on Sep 11, 2008, 03:47 pm
You may have noticed mellis has made me a moderator on the hardware interfacing forum where most of the spam has been. I've deleted a bunch already today.

While this isn't a long term solution it's better than what we've had.

I suggested it would be good to have at least one other moderator added.

Thanks mellis for organising this, it's greatly appreciated.

--Phil.
Title: Re: Spam fighting request
Post by: mem on Sep 11, 2008, 04:11 pm
Quote
You may have noticed mellis has made me a moderator on the hardware interfacing forum where most of the spam has been. I've deleted a bunch already today.

While this isn't a long term solution it's better than what we've had.

I suggested it would be good to have at least one other moderator added.

Thanks mellis for organising this, it's greatly appreciated.

--Phil.


Phil, the improvement was almost immediatly noticeable, thanks for volunteering. I will help out if no-one else steps up. It would be useful if the other mod was in a very different time zone to you, are you are in the US?
Title: Re: Spam fighting request
Post by: Oracle on Sep 11, 2008, 04:38 pm
Quote

Phil, the improvement was almost immediatly noticeable, thanks for volunteering. I will help out if no-one else steps up. It would be useful if the other mod was in a very different time zone to you, are you are in the US?


I did notice the improvement right away, even without knowing it was Phil at work.  It's so nice to go to that forum and not see spam as the top 20 posts.

I wouldn't mind helping, but you're right about the time zone issue and I am in Canada.
Title: Re: Spam fighting request
Post by: madworm on Sep 11, 2008, 05:51 pm
first let me thank the spam fighters for doing a fine job.

I happen to know that as of YaBB 2.2.3 it is possible to require admin approval of new registrations.
I'm not saying this will cut down spam to 0, but it would keep those characters from getting in at all.
right now it's just register, wait for the forum's email, click and you're in. ready to spam in less than 5 minutes.
I know this would shift some work onto the main admins (doing the upgrade of course), but it's worth a shot.
as far as I've red, this confirmation mode also requires the aspirants to write a short statement as to why they want to get in.
surely a monologue about the amenities of V 1 A G R 4 or similar will not work here :-)
Title: Re: Spam fighting request
Post by: follower on Sep 11, 2008, 06:30 pm
Quote
It would be useful if the other mod was in a very different time zone to you, are you are in the US?

That is a good suggestion. I'm in New Zealand--I guess that suggests perhaps someone in the East Coast of the US or in Europe. (Well, apart from the fact I don't exactly keep 9-5 hours. :))

--Phil.

P.S. Thanks all for the positive feedback. :-)
Title: Re: Spam fighting request
Post by: follower on Sep 11, 2008, 06:34 pm
Quote
I happen to know that as of YaBB 2.2.3 it is possible to require admin approval of new registrations.

I'm keen to minimise the impact on legitimate new (or existing) users. If things get "bad enough" I guess we could consider moderation of initial messages but my suggestion would be to see how long the "deletion regime" remains effective.

--Phil.
Title: Re: Spam fighting request
Post by: Oracle on Sep 11, 2008, 06:59 pm
Quote
I'm keen to minimise the impact on legitimate new (or existing) users. If things get "bad enough" I guess we could consider moderation of initial messages but my suggestion would be to see how long the "deletion regime" remains effective.


I tend to agree, if the sign-up process is too invasive, I generally don't sign up for a site.  One forum I'm on has an initial period where you can't post links, can't sent private messages, etc.  I think their threshold is 10 legit posts.  

But for now I'm just enjoying my spam-free Arduino experience, so thank you, Phil :).

And for the record, I'm in Eastern time in Canada ;)
Title: Re: Spam fighting request
Post by: madworm on Sep 11, 2008, 10:03 pm
Quote

I'm keen to minimise the impact on legitimate new (or existing) users


I wasn't talking about gagging new users, I just wanted to point out that with an update of YaBB to 2.2.3 it is possible to prevent this:

10: user applies for new account
20: current YaBB _automatically_ accepts that
30: user clicks on a link in the email
40: voila: valid account
50: SPAM SPAM SPAM
60: goto 10

if admins must confirm creation on new forum accounts, this process can be slowed down a lot.

I bet I could create accounts faster than any admin can delete them right now...
all I'm saying is that currently this forum is WIDE OPEN for spammers.
Title: Re: Spam fighting request
Post by: follower on Sep 12, 2008, 05:53 am
Quote
Quote

I'm keen to minimise the impact on legitimate new (or existing) users

(...snip...)

if admins must confirm creation on new forum accounts, this process can be slowed down a lot.

That's the impact I'm keen to minimise. :-) At the moment I would consider the impact of having new users wait outweighs the benefit of stopping a few spammer accounts. This could change in the future.  YMMV.

--Phil.
Title: Re: Spam fighting request
Post by: melka on Sep 12, 2008, 06:08 am
I'm ok to help follower on the de-spam task. I'm in Europe (France), and I check the forum at least each hour.
Cheers
Title: Re: Spam fighting request
Post by: follower on Sep 12, 2008, 04:06 pm
Quote
I check the forum at least each hour.
You're even more of an addict than me! :-)

--Phil.
Title: Re: Spam fighting request
Post by: mem on Sep 12, 2008, 06:41 pm
Quote
Quote
I check the forum at least each hour.
You're even more of an addict than me! :-)

--Phil.


Only once an hour, sigh! I think I need a 12 step program (written in C++ of course)   ;)
Title: Re: Spam fighting request
Post by: Oracle on Sep 12, 2008, 08:01 pm
Quote
[You're even more of an addict than me! :-)
--Phil.


I just read something about that.  Supposedly it's the same sort of addiction as a slot player.  Each time you visit it's like pulling the handle, and the jackpot is a good posting to reply to.
Title: Re: Spam fighting request
Post by: mem on Sep 12, 2008, 08:29 pm
Quote
Quote
[You're even more of an addict than me! :-)
--Phil.


I just read something about that.  Supposedly it's the same sort of addiction as a slot player.  Each time you visit it's like pulling the handle, and the jackpot is a good posting to reply to.
does that one count?
Title: Re: Spam fighting request
Post by: melka on Sep 12, 2008, 09:14 pm
Quote
Quote
Quote
[You're even more of an addict than me! :-)
--Phil.


I just read something about that.  Supposedly it's the same sort of addiction as a slot player.  Each time you visit it's like pulling the handle, and the jackpot is a good posting to reply to.
does that one count?

and now ?
Title: Re: Spam fighting request
Post by: follower on Sep 13, 2008, 03:09 am
LOL, yeah I've seen email and web browsing described in the same way.

Slightly back on topic, I've had a couple of PMs regarding the level of discourse in a recent thread.

I'm keen for us to maintain our current community "policing" of standards and not become the local deputy but leave that role to the existing global moderators.

I see myself as a moderator purely for anti-spam purposes--is that the current expectation?

--Phil.

P.S. *ka-ching*
Title: Re: Spam fighting request
Post by: John_Ryan on Sep 13, 2008, 03:25 am
The neighborhood is usually quite orderly. I think theres a single exception at the moment taking up space and it would be a shame if it became the norm.

Personally I'm not here often enough to notice the goings ons, but the spam however is evident each time I visit. So in order of priority, I guess spam would have to be at the very top.
Title: Re: Spam fighting request
Post by: westfw on Sep 18, 2008, 05:11 pm
The new YaBB seems to have some anti-spam measures, but I'm not sure that they're working right.  I got a "you're posting too fast, you might be a spammer" warning on using the PREVIEW function, and PREVIEW also seems to change the POST button to WAIT.  Also, after getting the warning and going BACK to continue/retry, I was stuck at a page where neither the PREVIEW nor POST buttons seemed to work any more.  (Copy, back to forum, retry-reply, paste, wait, post... worked ok.)
Title: Re: Spam fighting request
Post by: mellis on Sep 18, 2008, 08:06 pm
Yeah, I've been having trouble with them as well.  We will keep tweaking the settings.
Title: Re: Spam fighting request
Post by: madworm on Oct 31, 2008, 08:23 pm
What's with the quote button ?
Title: Re: Spam fighting request
Post by: gnu_linux on Nov 01, 2008, 12:31 am
Have you tried mod_sec?

:)
Title: Re: Spam fighting request
Post by: madworm on Nov 02, 2008, 07:08 am
No.

And if I were thick in the head I'd immediately ask why and how an apache module influenced said button.  ::)

Title: Re: Spam fighting request
Post by: gnu_linux on Nov 02, 2008, 06:28 pm
Sorry, that was for Mellis

:)
Title: Re: Spam fighting request
Post by: madworm on Nov 03, 2008, 02:04 am
I know  ;D

Just couldn't resist.
Title: Re: Spam fighting request
Post by: gnu_linux on Nov 03, 2008, 02:08 am
lol  ;D

let's make it official

@mellis: Have you tried mod_sec?

The server was having several issues last week
Title: Re: Spam fighting request
Post by: mellis on Nov 03, 2008, 03:13 pm
No, what does it do?
Title: Re: Spam fighting request
Post by: madworm on Nov 03, 2008, 04:26 pm
It is basically something "like" SELinux or AppArmour for apache. A bunch (actually many) core rules are provided for checking http requests, code insertion hacks and so on. Didn't check if it also provides some anti D.O.S. features. Seems like a bitch to configure. There's also a rule generator which scans logfiles and is supposed to learn what's normal, but again a lot of reading I think. Some of the docs talk about tomcat java application servers accessed by apache. Sounds like big business web servers to me.

p.s. Hey, this swear word filter just zapped "hctib" :-)
Title: Re: Spam fighting request
Post by: gnu_linux on Nov 03, 2008, 05:57 pm
@Mellis

mod_sec is like an application firewall for apache

When used correctly it can really cut down on forum spammers, injection attacks and all kinds of nasty things

/me recommends it :)