Pages: 1 [2] 3   Go Down
Author Topic: Where to send money for a new Forum Server?????  (Read 3293 times)
0 Members and 1 Guest are viewing this topic.
Left Coast, CA (USA)
Offline Offline
Brattain Member
*****
Karma: 361
Posts: 17294
Measurement changes behavior
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Yes, that is great news that:

Your aware of the problems
You have a plan
You do really care and hate that the site presently sucks
You have finally communicated to us about it.

We don't hate you, we just hate the problems  smiley-wink

Lefty
Logged

HOLLAND
Offline Offline
Full Member
***
Karma: 0
Posts: 178
HOLLAND!!!
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I really hope the server is getting upgraded because this stinks smiley-razz

even when I reply i have to refresh like 4 times to see my own post


I think that the choking is hapening by refreshing5 time's to see if there's a reply
Because if everybody does it its more likely that we get 10gb of traffic...

But offcourse i Would love to say a big [size=16]Thank you[/size] to the Arduino Team for trying to solve the problem !
We love you! ( At least i do :-[ smiley-wink )

BTW Would be quite nice to see a e-mail option when there is a reply
this would dramaticly reduce the hits and thuse the traffic

« Last Edit: October 22, 2009, 05:37:19 am by sasaa » Logged

London
Offline Offline
Tesla Member
***
Karma: 10
Posts: 6255
Have fun!
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
BTW Would be quite nice to see a e-mail option when there is a reply
this would dramaticly reduce the hits and thuse the traffic

there is a 'notify of replies' option available for each thread
Logged

Forum Administrator
MMX
Offline Offline
Edison Member
*****
Karma: 37
Posts: 1139
hallo kompis
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Ok guys,

the team here just made the decision to upgrade the service to get enough CPU to run the service while we keep on working on the forum's and wiki's updates.

The upgrade will happen during the next one or two days. Let's keep our fingers crossed to see it works fine. Our service will be interrupted for about one hour when the time comes.

Thanks for your patience and keep the discussion going on this thread,

/d
Logged

"The old Europe"
Offline Offline
Edison Member
*
Karma: 1
Posts: 2005
Bootloaders suck!
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Nice!

What about the DoS theory ?

Can it be normal to find numbers like this:

Search Engines: 61
MSN Search (59), Yahoo! Crawler (1), Googlebot (1)


Logged

• Upload doesn't work? Do a loop-back test.
• There's absolutely NO excuse for not having an ISP!
• Your AVR needs a brain surgery? Use the online FUSE calculator.
My projects: RGB LED matrix, RGB LED ring, various ATtiny gadgets...
• Microsoft is not the answer. It is the question, and the answer is NO!

Forum Administrator
MMX
Offline Offline
Edison Member
*****
Karma: 37
Posts: 1139
hallo kompis
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Nah!

no DoS, unfortunately we are all the ones to blame ... the community likes the forum and the wiki too much and that generates a lot of CPU overload. The conspiracy theory is down now.

Dave will be hiring more virtualization power for our servers, so that the 8 processors will not reach that incredible load level. For those of you interested in this matter, I am going to summarize the history of the Arduino servers a little:

- from the beginning we had an account at Site5, it was super cheap, and -as long as there weren't that many concurrent users things worked-. We had this account for almost 3 years

- about 1 year ago, when the downloads of the IDE made it to one every 10 minutes, we decided to move to ServInt, then the traffic suddenly multiplied by two. It was the proof that the service we had before wasn't enough ... the sad story is that we couldn't monitor this problem properly so we could never understand the size of the problem

- while on ServInt, and thanks to a better monitoring software we understood that downloads were becoming a problem ... we reached over 1TB transfers each month, therefore we decided to move the downloads to Googlecode, who kindly donated enough space to keep versions of the IDEs for the different operating systems

- taking away the downloads still means over 300GB of data transfer in the server. Again, once we freed the server from the files, a lot of the traffic started to show up on the forum. At the same time the blog jumped from 15.000 visits (not unique) to 150.000 per month

- so the new problem arose about two months ago, on "hot" weeks, we could reach over 15GB transfers daily on only requests to our forum (perl based) or our wiki (php5). This generates a bigger load in the CPU than serving static html files. We are talking that a perl page uses 7% of one CPU, while a php5 service from the wiki takes 4,5%, and a static page 0,0%. I am talking about real numbers I can see right now on the process list in the server. This means that 80 concurrent users on the forum will definitely block the traffic on the server. ServInt has been very kind allowing Arduino.cc to take over much more resources than we had when needed, but of course, now it is time to pay back :-)

The solutions have been explained a little earlier. We believe a combination of hiring more CPU power with improving the software on the server side is going to be the way to go around this. You will notice an immediate improvement in the next hours, while you will see the thing going stronger (hopefully) in less than a month. This will help us developing new services we have in mind like Web2.0 integration (e.g. Facebook, Flickr, Youtube or Twitter modules) in the wiki and forum, one-password-fits-all service, etc.

We have walked a long way until today, but I have the feeling, there is still a long path in front of us.

Thanks,

/d

PS. At this point, I have to thank many people for their support and comments during the last three years on how to handle the server. From Ladyada -who recommended ServInt first-, to Nick -who I got to know on the IRC channel and help me out setting up the mirror service, where you can find copies to the IDE back to version 4-. All your comments are taken into account and we will keep the thing going as long as there are users working and sharing.
Logged

North Yorkshire, UK
Offline Offline
Faraday Member
**
Karma: 104
Posts: 5531
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Yes, glad to have confirmation that it is being sorted.

It has been driving me a bit mad recently, it seems to have got even worse today...

I would have thought that with everyone being the 'programming' type, this forum would be the best server on the net but I understand now that that is not the issue  smiley-wink

Anyway, we are all happy to help if there is anything that we can do (as long as you don't say, don't use the forum, to decrease traffic!).  ;D

Mowcius
« Last Edit: October 22, 2009, 01:51:09 pm by mowcius » Logged

0
Offline Offline
Full Member
***
Karma: 1
Posts: 225
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The number of MSN bots is up to 85 now!  Are you sure we can't find a way to blame them?

Search Engines: 88
Yahoo! Crawler (1), MSN Search (85), Crawler (1), Googlebot (1)

Logged

North Yorkshire, UK
Offline Offline
Faraday Member
**
Karma: 104
Posts: 5531
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Members: 21
mowcius, glassman, RI now, Coding Badly, Sebi, madepablo, Darran King, whosawhatsis, Iteo, joe912, Osgeld, 33Warlord, AWOL, lalalandrus, Grumpy_Mike, Helmut, bill2009, flora, noelportugal, cjands40, walterr
Guests: 129
Search Engines: 75
Yahoo! Crawler (2), MSN Search (71), Crawler (1), Googlebot (1)

Well it's either MSN or all of them bloomin' guests  smiley-grin
Logged

Halifax, NS, Canada
Offline Offline
Full Member
***
Karma: 0
Posts: 117
void loop { }
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I've been running my own vBulletin forum for 6 years now (www.reality-check.ca) and if you need any help just let me know.  I can hook you up with an excellent web host.  My site is always fast.
Logged

Gallaecia
Offline Offline
Full Member
***
Karma: 0
Posts: 139
A TESLA LE ROBARON
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hola DAvid, te dejé un privado.
Perdón por escribirte aquí, pero es por si no te dabas cuenta de leerlo y ví que estabas siguiendo este post.

Un saludo.
Logged

New York
Offline Offline
Edison Member
*
Karma: 1
Posts: 1023
E != m*c^2
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Will we keep our post count after the software switch? I'm almost at god member  ;D
Logged

Spokane, Washington
Offline Offline
God Member
*****
Karma: 1
Posts: 686
My name is Bob, and I'm an addict.
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@Boots, have you been running for 3 years with the same traffic as Arduino.cc?  That'll make a big difference in speed..

But I agree, if there's anything us "users" can do to contribute, whether it be donate or whatnot, let us know!!:D There are plenty of users willing to work just as hard/much as the Admins to keep everything running smoothly!

But regardless, thanks alot for the great work thus far! We'll be here to trek through this.. "systematic problem"!
Logged

Halifax, NS, Canada
Offline Offline
Full Member
***
Karma: 0
Posts: 117
void loop { }
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I wasn't comparing my traffic to Arduino.cc, although in terms of forums, this isn't exactly a huge forum.  With a proper host it should be pretty fast.  
They should talk to MediaLayer.net ... I've been through a few hosts but I've been with these guys for the last 3 years and have nothing but praise for them.
« Last Edit: October 23, 2009, 08:34:21 pm by Boots » Logged

"The old Europe"
Offline Offline
Edison Member
*
Karma: 1
Posts: 2005
Bootloaders suck!
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Currently the forum runs faster than ever !
Logged

• Upload doesn't work? Do a loop-back test.
• There's absolutely NO excuse for not having an ISP!
• Your AVR needs a brain surgery? Use the online FUSE calculator.
My projects: RGB LED matrix, RGB LED ring, various ATtiny gadgets...
• Microsoft is not the answer. It is the question, and the answer is NO!

Pages: 1 [2] 3   Go Up
Jump to: