Thanks for the welcome and my apologies for taking a whiles to get back to you.
Documentation, I think it is always a balancing act between too much information and too little, for the target audience. I would prefer to see a little more detail but I am 40, not 14. I might, being kind, describe the Arduino docs as refreshingly sparse
The docs are too the point and they say, what they say, very clearly. In some ways, that makes the documentation better than having reams of distracting information to sift through.
Completely agree reflashing the WiFi shield is a pain in the behing and needs a method to verify which version is on the shield. There is a function in the library but mine is still returning V1.0.0 after I thought I reflashed it. I couldn't find any command for the Atmel DFU which would just report version either. I guess I am just going to have to fork out for an FTDI cable, just to check the version.
As for the WiFi shield. I very quickly lost any expectation that it would 'just work' after my first experiments. I could have blamed Arduino but instead, I sat back and had a little think about what I was asking my tiny Uno board to do. I don't think it is so much the shield might be shoddy, as cramming a Wifi stack, IP stack and DHCP, DNS clients onto an MCU friendly board, is a fundamentally difficult thing to do. When I bothered to think about it, it turns out implementing a web server on an MCU is a pretty big ask.
I can't really say why Arduino have arranged the hardware as they have but the hierarchal arrangement mirrors how network protocol stacks are implemented. In a PC or Mac, the WiFi adapter only looks after the datalink and physical layers, with the rest of the stack residing within software drivers and APIs. Even the cheapest of cheapest WiFi routers has much more RAM and a little more runtime available to it, than an MCU.
I had a quick scan through the code on the thread you posted. I don't know whether this will help you but it is just what has worked for me.
I had some issues with the board crashing when I tried to run the serial at top speed. I am guessing that the MCU is being starved of run time. The crashes stopped after I dropped the serial rate to 9600.
The reset button has locked me out ,once. When I looked at my code, it was entering a tight loop, polling a pin. A rearrangement of the code, to avoid needing the loop and all is well.
The WiFi shield has occasionally refused to start but only immediately after uploading a relatively large sketch. Press of the reset button and all is well. I am not losing sleep over it, because it is a failure mode which should never happen in production.
By far the most common cause of my code crashing, has been buffer overruns. If there is a 90 byte limit, I have yet to trip over it because I have been too busy learning to manage my Uno's memory more efficiently and dynamically.
The way the example web server sketch implements HTTP, is plainly wrong. It works as a proof of concept but enters the realms of undefined behaviour, as far as the HTTP spec is concerned. The example is all a bit confused really. It declares a response as HTTP/1.1 and proceeds to behave as HTTP/1.0
To be perfectly honest, implementing a web server is probably the worse first network project anyone could pick, because there is so little RAM to buffer requests and responses. HTTP is an extensible protocol but it is an extremely vague protocol because of it. It would be much easier to write a raw sockets application with the application protocol designed to fit the limitations of the device. That wouldn't suit what I have in mind though.