My apologies for my rant yesterday, I was out of order having a go like that. I know you are working hard, but it was incredibly frustrating as I was just trying to learn and read docs and I could see that so many things could easily be done to fix it. So instead of me just complaining, perhaps I can help a bit.
I've re-run that test and I can see that a lot of improvements have been made since then. However, there are still some simple things that would take minutes at most to help things.
The version of jquery used is 1.4.4 from 2010, over 2 years old, isn't very optimized and doesn't play well with modern browsers.
The version of mootools is from 2006(!!). Why would two overlapping frameworks be needed? jquery should do everything.
Because you're hosting jquery yourself AND browser caching isn't in use, every page load and every visit is loading 70k of (obsolete) jquery.
Let's say you have 10,000 page loads per day. Instead of hosting jquery yourself, let Google take the strain and take advantage of browser caching.
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
Remember you can always link to the current version by omitting the .version. eg:
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.8/jquery.min.js"></script>
will always give you the latest 1.8 version and
<script src="//ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"></script>
will roll you over to 1.9 when it's released if you feel brave!
Either way, that's 684Mb and 10,000 file-serves saved every single day just for changing 1 single line.
And if you HAVE to load mootools as well:
<script src="//ajax.googleapis.com/ajax/libs/mootools/1.4.5/mootools-yui-compressed.js"></script>
madbob:
lardconcepts:
In addition you could switch from Apache to something fast like Nginx, and the version of php is hopelessly out of date.nginx has been considered but actually is not an option: migration would require a large porting of current Rewrite rules, now massively used to glue together all the pieces. Not impossible, but requires some time.
Yes, it's a learning curve but it's SO worth learning. The difference between Apache+PHP mod and nginx+php5-fpm is unbelievable.
Take a look at this: Wordpress Performance Comparison: Using Nginx, Apache, APC and Varnish in Different Scenarios
At 40 concurrent users on his 512Mb VPS, his Wordpress starts to hit 1 second response times.
With Apache + PHP with APC and Varnish he's service 150 concurrent users with a 40ms response time.
That's nearly 4 times as many users getting served 25x quicker, all from the same box.
Definitely worth looking into! And the difference between mod_rewrite and conf files isn't actually that big once you get the syntax.
Here are my 3 conf files for Wordpress, phpbb3 and ZenCart.
There's nothing you can't do with Apache .htaccess that you can't do far more efficiently with Nginx conf directives. And the nginx forum and Stackoverflow will always help you if you get stuck.
The ideal is that the only hit you see in the log on the server which you are running php + mysql is 1 single hit to serve the dynamic page content.
That's not always possible, but Cloudflare will massively help toward that too.
madbob:
The very first and easiest free thing to do would be to use Cloudflare
I've observed bad performance of Cloudflare on at least one website far smaller than the Arduino's one, it impressed me quite negatively.
Did you had some positive experience?
I've be interested to know about this bad performance and what the conditions were.
Here's why I trust and use Cloudflare (and no, I don't get paid for saying this!):
"Last year, when a Web site launched by infamous hacker group LulzSec was knocked offline by a massive DDoS attack it turned to CloudFlare, a 30-person San Franicisco start-up for protection.
For the next 23 days, LulzSecurity.com stayed online, and CloudFlare, which had launched in private beta only 12 months before, withstood withering attacks from hackers around the globe.
The quality of CloudFlare’s service and its rock-bottom prices have been attracting new customers at the rate of 1,500 a day, Prince said, including Fortune 500 companies, major government agencies in the United States and abroad, and even sites owned by the White House. In two years, CloudFlare has grown so big that on any given day 25 percent of the Internet’s visitors pass through its network, Prince said. CloudFlare currently has servers running in 14 data centers around the world, including 7 in the United States, 3 in Asia and 4 in Europe."
And those figures are from Feb, so bound to be much higher. If it's good enough for 25 percent of the internet and the White House, it's good enough for me!
So, if you were to use Cloudflare, that takes care of the 20 static assets which should be being combined and minified.
Here's an example: I run 5 sites: 3 Wordpress, 1 busy phpbb forum and 1 horribly inefficient but busy old e-commerce system called Zen Cart - (80 db queries for the front page alone, nothing I can do about it sadly).
I run nginx, php5-fpm, apc, and varnish cache on the Wordpress sites, and Cloudflare sits in front of all of that. I run ALL of these sites from one single-core 1Gb RAM VPS costing me £10 a month.
Load rarely tops 25% according to "top", according to Google analytics my average total pageload time (all assets loaded) is 1.25 seconds, making all my sites faster than around 80% of comparable sites.
Here are my cloudflare stats for all 5 sites for the last 30 days:
51.1 GB bandwidth saved by CloudFlare
115.0 GB total bandwidth
3,844,627 requests saved by CloudFlare
5,232,005 total requests
777,088 Page views
581,657 regular traffic
95,391 crawlers/bots
100,040 threats
That's 100,000 pageviews saved from hackers even getting to your site.
And the advantage of Cloudflare is, if you don't like it, you login to the control panel and switch it off instantly.
Total time: 3 minutes to make a new Cloudflare account. 2 minutes to login to your registrar and change the name servers. (Plus propogation time of course!).