Update on WiFi shield 90 byte limit...

Hi,
I am using an Arduino Mega2560, and WiFi shield as an MQTT client, communicating with Mosquitto on a Linux box. I can send large amounts of data from Mosquitto to the Arduino, and all is OK.

Like many people, when I try to send more than 90 bytes from the Arduino, it fails.

I've installed Atmel studio, and downloaded the WiFi firmware (The 32U4 portion) from GitHub, and I can see the SPI polling code, which indeed uses a 100 byte receive buffer. The code is very close to the wire in terms of space on the 32U4, which is why, I would say, the developer(s) limited the buffer to 100 bytes.

I re-compiled the firmware under Atmel Studio 6, and produced a new .elf file, but when I flashed this to the WiFi shield, it wouldn't connect to the Mosquitto broker, (WiFi connection was OK), so I'm guessing that the source code on GitHub is out of sync with the latest firmware being shipped with Arduino 1.5.5.

So, I decided to alter the WiFiClient code to impose a 50 byte MTU limit for the SPI transmission from my Arduino Mega to the Shield. This works, up to a point. I successfully transmittefd 200 bytes just fine.

However, last thing last night I tried to send a 500 byte message from a file, and crashed the shield. I'm guessing another buffer over-run. More tonight.

Some questions, for anyone who is working in this area;

  1. Has anyone else overcome this issue?
  2. Does anyone know where I can get the latest source code for the 32U4 firmware?

My next step is to try the CC3000 shield. Unfortunately, a quick look seems to indicate that the software library for this shield does not inherit from Client, so I will have to write a wrapper class for it in order to use it with the Arduino MQTT Client, which expects a Client.

Is anyone else still working on this?

Cheers,
Con

Where I got to with the WiFi shield.

Summary: The shield is usable for sending/receiving small UDP packets most of the time, but not much else.

Issues:

  • The 90 byte transfer limit is annoying but not game breaking, Throughput appears to be bottlenecked further by a polling loop in the order of seconds.
  • During long running UDP tests (days), some transient event I have yet to identify, places the shield into a stalled state, with none of the software gymnastics I have tried, able to recover or restart it. The transient event may have something to do with roaming across access points and expiring authentication tokens.
  • The TCP implementation is completely borked, from the layer 7 Sockets implementation, down to the sequencing of SYN/FIN/RST/ACK packets at layer 4. The shield will just about work as a TCP client, providing the server can tolerate orphaned sessions.

The state of the firmware leaves the shield being too broken to be much use to me. Fixing the firmware is beyond the time, knowledge and resources I have available. So the very best of luck with it.

I have since acquired a Yun.

I have not experienced the UDP fail as Matt describes, but I probably am not testing it to its limit there. Here my test:
http://forum.arduino.cc/index.php?topic=189122.0
I finally stopped that test at about 180,000 packets. I needed my test Arduino for another test.

I haven't tried any files over 250K, but those did ok. I send them in 64 byte packets, and it does fine. I use the same on the ethernet shield without problems. However, this is in a controlled environment. If the wifi shield is exposed to the internet and hackers, it will crash. On a localnet with one client, it does fine. That is the test I am running now. It has been running about 5 days now without a crash.

My big complaint is the lack of interest in the wifi shield by the Arduino team. They seem to be busy with the newer models to fix any of this.

Hi Matt,
I dumped the Yun - the "Bridge" between the 32U4 side and the Linux SOC is just too kludgy - they just glued the hard console from the SOC to the UART on the 32U4.

I have a Linux hub, and I just want a reliable means of talking from that to a bunch of nodes which control hot water, heat,etc. I want the nodes to be as reliable and autonomous as possible, ie just my own bare-metal code, so I chose Arduino. MQTT is the protocol for comms, so my iOS and Android speak MQTT to the Linux box (running MQTT broker - Mosquito) and the Linux box broadcasts MQTT messages, which the Arduino nodes listen for and respond to.

So far, I've been through Yun, (Didn't like the bridge machanism), Xbee (Just a mess really), WiFly shield (Doesn't work with the Mega2560, which I need for program space), and now the orphaned and abandoned WiFi shield. I agree the firmware is a mess, and nobody seems to take responsibility for it. I know this is all open source, but if someone makes money on a shield, they should support the firmware.

I've ordered a C3000 shield from AdaFruit, and will wrap that as an Ardiono CLient class, and try that next. I will report on how it goes here.

Meantime, just for pig-iron, I've ordered an FTDI cable, and am going to try to find out why the WiFi Shield is crashing. I can't send anymore than 250 bytes, in 50 byte chunks, using the WiFiClient class. SurferTim, did you use WiFiClient to send the large files to a server? If so, did you have any delays between the 64byte chunks?

Cheers,
Con

I used a server sketch to send the files to a client. It uses the client functions to send it though. No delays.

GreyCon:
I have a Linux hub, and I just want a reliable means of talking from that to a bunch of nodes which control hot water, heat,etc. I want the nodes to be as reliable and autonomous as possible, ie just my own bare-metal code, so I chose Arduino. MQTT is the protocol for comms, so my iOS and Android speak MQTT to the Linux box (running MQTT broker - Mosquito) and the Linux box broadcasts MQTT messages, which the Arduino nodes listen for and respond to.

At current moment I'm running almost the same, but instead of MQTT broker I'm using Domoticz on Raspberry.
It's open source and people on forums are working with MQTT integration to it.

My current interface is MEGA with RF24(Network) nodes all over house.
I'd like to (if it's possible) to look @ your code and maybe find some inspiration with switching to complete-MQTT with nodes and MEGA.

The RF24 limitation is 24 bytes in one message, I'm not that good with thinking how to program it to get messages queued od sent-together other way...

Hi Andriej,
change of plan! I've decided to keep the arduino as a very dumb "Bare metal code" device, which simply stores timers in EEPROM, and using a real time clock, operates a relay to do switching. It also reads temperature values from the tank using DS18B20 sensors.

I've setup a Raspberry Pi as the hub now, running Mosquitto, and a BeagleBone black as the node. I am connecting the BeagleBone black to the Arduino, using a serial link. Thus, if the BeagleBone ever dies, the arduino will continue life as an autonomous timer. (Main design goal.) But, the BeagleBone allows me to write lots of code to handle MQTT messages, parse the JSON data, store historic temperature values etc and supply these back via the Pi Hub as and when needed.

It's over engineered, but a good solution. It's essentially the same design philosophy as the Arduino Yun, but with a bit more oomph on the Linux side, and more control over the connection between the Linux and the Arduino side.

I'm woking on the code for this new setup at the moment. I'm not sure if any of it is any use to you, but if you think it might be, just shout - you're welcome to it.