Maximum file serving speeds capable by arduino? (TCP/FTP/WiFi/Bluetooth/RS232)

Hi everyone

I'm very interested to know the maximum transfer speeds that have been achieved by Arduino acting as a file server using the various network protocols such as RS232 vs Bluetooth vs TCP vs FTP vs Wifi. If someone has done a proper comparison I'd love to see it.

I've been scouring the net over the last few weeks and have found various webserver examples, and implemented a few myself, but I still can't seem to get the transfer speeds I need. The BEST rate I've got so far is 80kB/s via ethernet webserver, and about 2kB/s via WiFi. I've read of people getting 240kB/s via ethernet on custom light FTP servers so I know it's got to be possible to improve my speeds. I'm guessing it's my newb SPI and buffering code that might be inefficient. I've outlined some summary info below. I'd really appreciate any input on this from the pros!

And just to clarify - the arduino is the file server not the client for this scenario, and is serving files from the SD card to a client computer.

My Setup Ethermega (ATMega256) SD Card (via SPI) ATWINC1500 WiFi (IEEE 802.11 b/g/n running on SPI). File Serve Method: HTTP Webserver accessed by a client browser (Console access for RS232).

MAXIMUM SPEEDS ACHIEVED

RS232: 14kB/s Via console I can serve large files direct from the SD card at around 14KB/s at 115200 baud (which is essentially the bit rate in short distance digital serial comms). As expected.

Bluetooth: ? Not tested yet. I understand the bluetooth protocol supports up to 50Mbit/s in theory, however in reality Bluetooth 4.1 you can get about 1Mb/s (125kB/s) and the upcoming Bluetooth 5 is touted for 2Mb/s (333kB/s) max throughput (wikipedia). I suspect performance may also be severely constrained on a microcontroller like arduino. I'm guessing you might double 114200baud speeds to say 30kB/s?

HTTP webserver - Ethernet (TCP/IP): 80kB/s I implemented Sean Ashton's WebDAVNav web based file server and modified it to server HTML not XML. Got it serving files from the SD card at 80kB/s. Tried to get sirleech's Webduino server going to no avail.

HTTP webserver - WiFi (TCP/IP): 2kB/s I modified standard WiFi Webserver and coded SPI handling to share the SPI Wifi and SD card to serve files via HTTP GET method. It worked but is horrendously slow. Suspect my buffering code. I also looked at the Jaycar example of a WiFi webserver, however it uses the older style WiFi shield that uses serial, not SPI. So it's limited to serving at 500bytes/second.

Ethernet FTP fileserver (TCP/IP): ? Not tested. I haven't implemented a light FTP server. I've read of people claiming 255kB/s transfer speeds using a Due. This is impressive. Also note another few FTP file servers I found: https://github.com/mihaigalos/FTPino http://github.com/gallegojm/Arduino-Ftp-Server

Summary Basically I'm trying to get the best possible wireless transfer speeds, preferably by a simple WiFi HTTP file server, so the client browser can rapidly download files from the arduino SD card.

Thanks everyone :) Dan

My experience has been the limiting factor has been the SPI bus, and the sending of one byte per packet if your code is not efficient.

Ok thanks. Given that the Ethernet controller and Wifi both need to share the SPI bus with the SD card, I don't imagine there would be much difference between the two, in terms of max speed as the bottleneck is the bus not the protocol speed.

I'm going to try to modify a light FTP server built for ethernet and modify it for Wifi. In theory I think this can be done but I'm really not sure if the hardware can support it. One issue I've heard with FTP that causes problems is the need for two connections (Data and Command).

FTP does require two connections. You might want to look into TFTP. Somebody here got it to work. Only requires one connection. I didn't try it. I migrated my program to a RPi.

Thanks SurferTim

I’ve seen your name quite a bit during my research - pretty sure you’ve written your own FTP server :slight_smile:

Will take a look, but hoping I can get better performance out of a simple HTTP GET request and serve files via wifi straight to a client browser. No software required. HTTP can be just as quick as FTP for this application, so I think my issue is tweaking the buffering/spi code, which I’ll do by looking at how it’s been done in the ethernet server sketches. Pretty sure I’m just using retarded buffering, like issuing 1 character at a time instead of the whole 512 byte buffer, etc.

Unfortunately I have to stick with ATMEGA256 as this application is for embedded sensors that are battery operated, Pi is just too big a rig us.

If you have any performance stats on FTP though Tim I’d love to know.

Cheers
Dan

Bump :)

UPDATE:

Managed to get WiFi serving files to a web client at 20kB/s.

This is 10 times faster than before but still far below the 80-250kB I've seen on ethernet examples. If there are any gurus that can look at my main loop code below and identify any obvious bottleneck that would be great.

Note I'm sharing the SD card and Wifi via SPI, so have manually controlled this in my file read loop. So I open a file, read a 1024 byte buffer. Chip select SPI to to the Wifi, write the buffer, then switch back to SD card and keep reading the file.

I have tried: 1. Fast digital/write using low level C commands (PORTG &= ~_BV(PG5); //SD on). 2. SPI_FULL_SPEED option at init of the SD card. This actually slowed it down to 8.5kB/s 3. Clearing the 1024byte buffer using memset instead of a slow for loop one char at time.

I'm now officially stumped. I think the bottleneck might be the rate I can read the SD card. Just not sure.

My main 'file serving' method that runs on the arduino, serving a web client. HTTP header info already sent:

void downloadfile(WiFiClient cl) { Serial.println("Attempting to send file from SD card to web client");

PORTG &= ~_BV(PG5); //SD on PORTH |= _BV(PH5); //wifi off //delay(50);

if (SD.exists(FILENAME)) { Serial.print("log file "); Serial.print(FILENAME); Serial.println(" found"); } else { Serial.println("log file not found"); return; }

Serial.println("Opening log file..."); fhandle = SD.open(FILENAME, FILE_READ);

//IMPORTANT LOOP CODE STARTS HERE counter1=0; //reset buffer counter if (fhandle) { //file open while (fhandle.available()) { //if a byte can be read

Buffer[counter1]=fhandle.read(); //read byte of data to buffer BufferLen = counter1;

//check buffer overflow if (counter1+1 > MAX_BUFFER){ Serial.println("Buffer Overflow. Program terminating."); counter1=0; return; }

//check to see if buffer full if (counter1+1 == MAX_BUFFER) { //Send buffer to client browser and continue //now we want to push buffer to client and read next 3kb of data from file if availble PORTG |= _BV(PG5); //SD off PORTH &= ~_BV(PH5); //WF on cl.write(Buffer, BufferLen); //write the buffer to wificlient memset(Buffer, 0, BufferLen); //reset buffer using memset BufferLen=0; counter1=0; PORTG &= ~_BV(PG5); //SD on PORTH |= _BV(PH5); //wifi off

}

counter1++ ;

} //end while byte available

//Serial.println("no more bytes available"); //note at this point the file may have closed naturally, and we may need to send whatever remains in buffer to file. PORTG |= _BV(PG5); //SD off PORTH &= ~_BV(PH5); //WF on cl.write(Buffer, BufferLen); //write anything left in the buffer //Serial.println("Buffer written, now clearing buffer and turning SD on"); //clear buffer and reset counters memset(Buffer, 0, BufferLen); BufferLen=0; counter1=0; PORTG &= ~_BV(PG5); //SD on PORTH |= _BV(PH5); //wifi off

// close file to exit while Serial.println("Copy finished, closing file"); fhandle.close();

} //end if file open

//Serial.println("Turn off sd and turn on wifi");

PORTG |= _BV(PG5); //SD off PORTH &= ~_BV(PH5); //WF on

}

Can anyone help??

This is what I use in my server example.

// in local data
   char tBuf[64];
   int clientCount = 0;

// then send a file
                File myFile = SD.open(fileName);

                if(myFile) {
                  while(myFile.available()) {
                    clientCount = myFile.read(tBuf,64);
                    client.write((byte*)tBuf,clientCount);
                  }
                  myFile.close();
                }

Thanks Surfer Tim.

I see you are using a buffer of 64 bytes which I gather is optimal as it matches the standard max TCP/IP packet size.

Thanks for your code snippet - I think I've found my problem. I'm sending the 1024 byte buffer in one write command, BUT I'm reading in the buffer from the file at a single byte at a time (retarded).

Do you happen to know the max speed you've clocked your ftp file server at? :)

Did you manage to get it to work?

I am attempting to do the same and could use some help.

Thanks

W Daum