Ethernet Shield Throughput Measured

I'm currently evaluating the Arduino platform for potential use in product development, but I couldn't seem to find any real specs regarding throughput. One poster in another topic commented about something like 5KB/S, but there was nothing concrete. So, I purchased an Ethernet shield and ran some simple tests.

The Arduino used for the test is the Mega 2560. The Ethernet shield is a clone.

The sensor reading web server page was used example code was used to perform the test with the addition of two lines of code to set a pin high just prior to sending the web page and then low once the page was sent out. This line was connected to a DSO to determine the timing.

Now, keep in mind that this is the time it took for the micro-controller to hand off the web page to the Ethernet shield. The W5100 chip adds in the necessary overhead to transmit the data over TCP/IP protocol. However, even on a 10Mb Ethernet connection, the data transfer is still considerably faster than the rate that the Arduino can send the data through the SPI interface. So these measurements still reflect the slowest link in the chain, so to speak, and should serve as reasonable benchmarks.

Using the standard Ethernet library, and the example web page, the first measurement worked out to about 15KB/S. This is certainly better than the 5KB/S off the cuff rate that I had seen elsewhere. But it seemed pretty slow and the hardware should be able to do better.

The A/D measurement function was commented out, since I remembered that process can take some time. This increased the rate to 17.5KB/S and fixed the number of bytes being sent to the Ethernet shield to 273 (if I counted correctly). Still, an improvement.

At that point I added another connection to the DSO, attempting to see just what was happening on the SPI interface. The first thing that jumped out was that the clock was running at 4MHZ. With the Mega 2560, 8 MHZ is supported. Changing the rate means adding a line in the W5100.CPP file to use the setClockDivider() function. This rate is not supported on all Arduinos and I don't recommend doing this change in the standard library. For experimentation this was the simplest way to test the higher clock rate.

Increasing the clock rate increased throughput to 23.9 KB/S, about 27% improvement. Still curious, more scope measurements were done. Based on the captured waveforms, it looked like a handfull of bytes were being sent to the W5100, then a rather large delay would take place, then some more bytes were going out. Examining the library code, no specific or intentional delays could be found. Eventually, it seemed like each group of bytes was about as long as a single line of text, leading to the thought that the print() and println() functions were slowing things down.

An alternate function is available for sending data, the write() function. This function can only accept an array of characters (actually it is a uint8_t pointer, but that isn't terribly important at the moment). Changing the first line into an array of characters nudged the speed up to 24.5 KB/S. Changing all of the lines into character arrays increased the speed to 70 KB/S!

So, the single most important thing that you can do to increase Ethernet throughput is to avoid using print and println. The changes go something like this:

client.println("HTTP/1.1 200 OK");

becomes this:

byte l1[] = {'H','T','T','P','/','1','.','1',' ','O','K',0X0D,0X0A};
client.write(l1,sizeof(l1));

Note the added bytes at the end of the array. You must add line termination yourself.

This technique only works with static strings. To send variable data, a byte array must be populated with the data to send. Numeric values must be converted to string characters. This is definitely an advanced technique, so if you don't have a good understanding of C++, I'd recommend not attempting this. But if you really need to speed up sending speed, this is the only way to do it.

As far as getting more speed out of the Ethernet shield, there are some specific limitations that can only be addressed by adding custom hardware.

For each data byte that needs to be sent, a command consisting of 4 bytes are sent over the SPI bus to the W5100 chip. There are small, but cumulative delays between each of those command bytes, 600 nS between bytes for a total of 1.8uS per actual data byte that you want to send. These delays are due to the time it takes the code to determine that the last byte send of SPI is complete, to get the next byte from the data buffer, and to place that byte in the SPI data register to be sent. This delay works out to about 5 processor instruction cycles.

Delays also show up between each of these command byte groups, anywhere from 2.3 uS to 5.9uS was measured with this setup. This delay works out to between 18 to 47 processor instruction cycles.

These delays are unavoidable. So, for all practical terms, you can get up to 70KB/S with an Mega2560 if you increase the SPI clock speed and avoid using the print() and println() functions.

Thank you, that is very usefull.
Perhaps a link to your post should be added to the 'Ethernet' section in the Playground.
http://playground.arduino.cc//Main/InterfacingWithHardware#ethernet
But that section is confusing.

The mentioned 5kbyte/s is when reading a htm file from the microSD card.
That can be optimized to 15kbyte/s with a buffer for the microSD data.

Are you talking about 70 kilo bits per second ? That would be 9 kilo byte per second ?

The normal way to send a data of a string could be strlen or size - 1 ?

client.println("HTTP/1.1 200 OK");

becomes this:

byte l1[] = "HTTP/1.1 200 OK\r\n";
client.write(l1,sizeof(l1)-1);      // don't sent the zero terminator of the string

70 Kilobytes per second.

Yes, the string assignment you are doing is much easier. I haven't been on the C++ side of the cubicle in many years.
And when using a direct string assignment like that, you do need to do sizeof() - 1 to strip off the 0 character.

Thanks for pointing that out.

I have a complete website on the microSD card with favicon.ico and small pictures and subdirectories. But at the moment I don't have to time make a profile of the Ethernet and microSD card SPI timing.

Server test code that speeds up an upload of a file from the SD card.

//zoomkat 12/26/12
//SD server test code
//open serial monitor to see what the arduino receives
//address will look like http://192.168.1.102:84 when submited
//for use with W5100 based ethernet shields

#include <SD.h>
#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 
  192, 168, 1, 102 }; // ip in lan
byte gateway[] = { 
  192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 
  255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port
String readString; 

//////////////////////

void setup(){

  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);
  Serial.print("Starting SD..");
  if(!SD.begin(4)) Serial.println("failed");
  else Serial.println("ok");

  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  //delay(2000);
  server.begin();
  Serial.println("Ready");
}

void loop(){
  // Create a client connection
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {
          //store characters to string 
          readString += c; 
          //Serial.print(c);
        } 
        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.println(readString); //print to serial monitor for debuging 

            client.println("HTTP/1.1 200 OK"); //send new page
          //client.println("Content-Type: text/html");
          client.println("Content-Type: image/jpeg");
          //client.println("Content-Type: image/gif");
          //client.println("Content-Type: application/x-javascript");
          //client.println("Content-Type: text");

          client.println();

          //File myFile = SD.open("boom.htm");
          File myFile = SD.open("HYPNO.JPG");
          //File myFile = SD.open("BLUEH_SL.GIF");
          //File myFile = SD.open("SERVOSLD.HTM");

          if (myFile) {

            byte clientBuf[64];
            int clientCount = 0;

            while(myFile.available())
            {
              clientBuf[clientCount] = myFile.read();
              clientCount++;

              if(clientCount > 63)
              {
                // Serial.println("Packet");
                client.write(clientBuf,64);
                clientCount = 0;
              }
            }
            //final <64 byte cleanup packet
            if(clientCount > 0) client.write(clientBuf,clientCount);            
            // close the file:
            myFile.close();
          }
          delay(1);
          //stopping client
          client.stop();
          readString="";
        }
      }
    }
  } 
}