Website very slow

No offence intended, but this site urgently needs someone who knows how to run a webserver to get involved.

I've never seen a site fail so badly on so many points - WebPageTest Test - Running web page performance and optimization tests...

There are LOADS of suggestions there that could easily be implemented and would massively speed the site up.
In addition you could switch from Apache to something fast like Nginx, and the version of php is hopelessly out of date.
The very first and easiest free thing to do would be to use Cloudflare; that would take at least 50% of the load of the server and do a lot of optimisation.
Then you could add in Varnish cache - end result: 200ms page response times rather than 20seconds, net cost: half a day of the time of someone that knows what they're doing. No need to spend money on new servers.

But in short, this site isn't suffering because of visitor count, it's suffering because it's hopelessly unoptimised and urgently needs some TLC.

Today's problems have been mostly caused by the huge traffic drived by the latest post on our blog

For all the day the load average on the server has been above 200, and our 8 processors never stop to run.

That permitted to put in evidence large optimization margins still existing on this server: discrete performance tuning has been constantly in place in last two months, targetting above all the forum, but other major portions of the whole Arduino web platform still needs some revision.

lardconcepts:
In addition you could switch from Apache to something fast like Nginx, and the version of php is hopelessly out of date.

nginx has been considered but actually is not an option: migration would require a large porting of current Rewrite rules, now massively used to glue together all the pieces. Not impossible, but requires some time.

The very first and easiest free thing to do would be to use Cloudflare

I've observed bad performance of Cloudflare on at least one website far smaller than the Arduino's one, it impressed me quite negatively.
Did you had some positive experience?

Then you could add in Varnish cache

Yes, this is already on my todolist :wink:

Sorry for the bad service seen today, your sysadmin is working for you.

Response is much better now, FWIW.

I couldn't even log in at work today to respond to anything. My entries were accepted - and then nothing, was treated just as guest.

My apologies for my rant yesterday, I was out of order having a go like that. I know you are working hard, but it was incredibly frustrating as I was just trying to learn and read docs and I could see that so many things could easily be done to fix it. So instead of me just complaining, perhaps I can help a bit.

I've re-run that test and I can see that a lot of improvements have been made since then. However, there are still some simple things that would take minutes at most to help things.

The version of jquery used is 1.4.4 from 2010, over 2 years old, isn't very optimized and doesn't play well with modern browsers.
The version of mootools is from 2006(!!). Why would two overlapping frameworks be needed? jquery should do everything.

Because you're hosting jquery yourself AND browser caching isn't in use, every page load and every visit is loading 70k of (obsolete) jquery.

Let's say you have 10,000 page loads per day. Instead of hosting jquery yourself, let Google take the strain and take advantage of browser caching.

 <script src="//ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>

Remember you can always link to the current version by omitting the .version. eg:

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.8/jquery.min.js"></script>

will always give you the latest 1.8 version and

<script src="//ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"></script>

will roll you over to 1.9 when it's released if you feel brave!

Either way, that's 684Mb and 10,000 file-serves saved every single day just for changing 1 single line.

And if you HAVE to load mootools as well:

<script src="//ajax.googleapis.com/ajax/libs/mootools/1.4.5/mootools-yui-compressed.js"></script>

madbob:

lardconcepts:
In addition you could switch from Apache to something fast like Nginx, and the version of php is hopelessly out of date.

nginx has been considered but actually is not an option: migration would require a large porting of current Rewrite rules, now massively used to glue together all the pieces. Not impossible, but requires some time.

Yes, it's a learning curve but it's SO worth learning. The difference between Apache+PHP mod and nginx+php5-fpm is unbelievable.
Take a look at this: Wordpress Performance Comparison: Using Nginx, Apache, APC and Varnish in Different Scenarios

At 40 concurrent users on his 512Mb VPS, his Wordpress starts to hit 1 second response times.
With Apache + PHP with APC and Varnish he's service 150 concurrent users with a 40ms response time.

That's nearly 4 times as many users getting served 25x quicker, all from the same box.

Definitely worth looking into! And the difference between mod_rewrite and conf files isn't actually that big once you get the syntax.

Here are my 3 conf files for Wordpress, phpbb3 and ZenCart.

There's nothing you can't do with Apache .htaccess that you can't do far more efficiently with Nginx conf directives. And the nginx forum and Stackoverflow will always help you if you get stuck.

The ideal is that the only hit you see in the log on the server which you are running php + mysql is 1 single hit to serve the dynamic page content.
That's not always possible, but Cloudflare will massively help toward that too.

madbob:

The very first and easiest free thing to do would be to use Cloudflare

I've observed bad performance of Cloudflare on at least one website far smaller than the Arduino's one, it impressed me quite negatively.
Did you had some positive experience?

I've be interested to know about this bad performance and what the conditions were.

Here's why I trust and use Cloudflare (and no, I don't get paid for saying this!):

"Last year, when a Web site launched by infamous hacker group LulzSec was knocked offline by a massive DDoS attack it turned to CloudFlare, a 30-person San Franicisco start-up for protection.

For the next 23 days, LulzSecurity.com stayed online, and CloudFlare, which had launched in private beta only 12 months before, withstood withering attacks from hackers around the globe.

The quality of CloudFlare’s service and its rock-bottom prices have been attracting new customers at the rate of 1,500 a day, Prince said, including Fortune 500 companies, major government agencies in the United States and abroad, and even sites owned by the White House. In two years, CloudFlare has grown so big that on any given day 25 percent of the Internet’s visitors pass through its network, Prince said. CloudFlare currently has servers running in 14 data centers around the world, including 7 in the United States, 3 in Asia and 4 in Europe."

And those figures are from Feb, so bound to be much higher. If it's good enough for 25 percent of the internet and the White House, it's good enough for me!

So, if you were to use Cloudflare, that takes care of the 20 static assets which should be being combined and minified.

Here's an example: I run 5 sites: 3 Wordpress, 1 busy phpbb forum and 1 horribly inefficient but busy old e-commerce system called Zen Cart - (80 db queries for the front page alone, nothing I can do about it sadly).

I run nginx, php5-fpm, apc, and varnish cache on the Wordpress sites, and Cloudflare sits in front of all of that. I run ALL of these sites from one single-core 1Gb RAM VPS costing me £10 a month.
Load rarely tops 25% according to "top", according to Google analytics my average total pageload time (all assets loaded) is 1.25 seconds, making all my sites faster than around 80% of comparable sites.

Here are my cloudflare stats for all 5 sites for the last 30 days:

51.1 GB bandwidth saved by CloudFlare
115.0 GB total bandwidth

3,844,627 requests saved by CloudFlare
5,232,005 total requests

777,088	Page views
581,657	regular traffic
95,391	crawlers/bots
100,040	threats

That's 100,000 pageviews saved from hackers even getting to your site.

And the advantage of Cloudflare is, if you don't like it, you login to the control panel and switch it off instantly.
Total time: 3 minutes to make a new Cloudflare account. 2 minutes to login to your registrar and change the name servers. (Plus propogation time of course!).

lardconcepts:
So instead of me just complaining, perhaps I can help a bit.

Pleased to see your collaboration :slight_smile:

The version of mootools is from 2006(!!). Why would two overlapping frameworks be needed? jquery should do everything.

Good question, the response is "I don't know". Mootools seems required by the current SMF's theme but it is unclear to me the usage, I will try to disable it on our development instance and see what it happens.
About using Google's CDN for jQuery: you are absolutely in reason, the test for migration to 1.8 is in my schedule (since far too long time, really).

I've be interested to know about this bad performance and what the conditions were.

I've observed the experience of a friend of mine, on a small media coverage website with low traffic (peaks of 50 concurrent visitors).
Too often the HTTP request failed in an error by Cloudflare complaining it was unable to find the target content, and very little hints was provided to correct the issue.
Not a so good impression...

This morning I began to put a cache plugin on the Wordpress instance driving the Arduino blog, just to provide a quick and dirty solution to the still moderately high traffic driven by the latest post published. And the Varnish documentation is opened on one of my browser tabs, ready to be implemented on the development server in next days.

I can stand the forum being slow (for a while) as long as it is obvious that someone is at least making an effort to do something about the situation. Thank you for that. Keep us posted on your efforts, please.

This same situation (slow slow response time) has happened at least three other times in my 3+ years on this site. Each of those times it did take some time for someone to actually do and/or change something fundamental (new server, new hosting software, etc) to improve the site's basic performance. So whether it's the continued growth of the site or something else, the site does seem to need some careful care and feeding again as it has needed in the past.

Thanks for any help that can improve things here.

Lefty

PaulS:
Keep us posted on your efforts, please.

Improvement of the day: PmWiki's FastCache has been enabled on Arduino Playground also (on the main Arduino website it was already in place).

Current load average of the server is between 7 and 10, quite good compared to the common 20 and the 240 peak of two days ago.

Today we hit a new low in server responsiveness, but cause is unknow: traffic was the same as always, no major content was published and visited, no anormal processes were running... I'm still reading logs to find a reason...

Also, I spent just a little time in reading Varnish documentation and testing elsewhere: still some issue about cache management, but I think next week I can run it on main server. All of you will be alerted and asked to report me about undesired behaviours, I expect your collaboration :wink:

I'm happy to notice the performance is actually actively monitored
Best regards
Jantje

All of you will be alerted and asked to report me about undesired behaviours, I expect your collaboration

If it doesn't time out trying to post... 8)

madbob:
All of you will be alerted and asked to report me about undesired behaviours, I expect your collaboration :wink:

Here we go: Varnish is actually in action.

Connections from authenticated users are passed to the backend, but all traffic from occasional users (e.g. people landing to the forum from Google) are managed by the cache.

If you encounter problems, errors, unexpected behaviours and so on, drop me a line in this thread or via webmaster@arduino.cc

Thanks for support and further suggestions.

Just wanted to add that the site has been very responsive over the last week. Significant improvement over a couple of weeks ago.

I can actually read the forums in under an hour!

codlink:
Just wanted to add that the site has been very responsive over the last week. Significant improvement over a couple of weeks ago.

I can actually read the forums in under an hour!

Yes site performance as been good for awhile now, but today I noticed that all the member's avatars are not displaying, what kind of outfit is running this show anyway? :smiley:

Lefty

I have just noticed that if you try to click on a subject in the old forum, you will be redirected to the new forum.
Actually i found out when doing a google search that had a link to the old form.

CrossRoads wrote:
I couldn't even log in at work today to respond to anything.

It's a very bad day when it won't even let you in!!

It's also been slow for me many many times in the past couple of weeks.

Forum is still very slow these last few days. May just be a Canada-Italy thing as other sites don't seem to be suffering from the same lag.

Mornings are okay for me but come 15:00 the site turns to treacle. Not sure if this is kids getting out of school or people across the pond waking up.

Generally I am finding the response times have improved somewhat, but then there are still a number of times where I just get a timeout or enough time to make a cup of tea.
From all the sites I use, this one still does appear to be problematic, which is a shame as it is one of my most used sites and forums theses days.

But I do appreciate the effort you are putting into it recently, I can understand it is time consuming maintaining these servers and their performance.

Paul