Hi,
I'm thinking about a project, that needs to receive .wav audio files from a remote host via Ethernetshield.
Those files should be stored on a SD card and be processed later by the adafruit WaveShield.
Is this possible? Does someone have an idea how to do this?
Maybe it is possible to sync with a FTP server?
What data transfer rates can be achieved with an Arduino Ethernet connection?
Thanks for any suggestions.
frsc
Most easy is probably to have the Arduino make a HTTP Get request, although implementing an FTP client should also be feasible. Just make sure your packets don't become too big, because memory is really tight on the Arduino.
For the data rate, you're basically limited to serial modem speeds. Don't expect more than 38400 bps if the Arduino is also doing other stuff.
Korman
Thanks!
Do you know some code examples where something similar is done?
I know how to make a simple http get request from various examples.
But how can I stream the packets to the SD card?
Basicly I want to do the very same as the questioner here: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1295493614
But how can I stream the packets to the SD card?
With no code posted, we have no idea what you are capable of. Do you know how to set, and send to the Serial Monitor, the data returned by the server in response to the GET request?
Do you know how to write to the SD card?
Do you know how to switch between using the SD card and the Ethernet shield? Are you even aware that you can't use them simultaneously?
Reading data returned by the server, switching between the Ethernet shield and SD card, writing to the SD card, and switching back to the Ethernet shield are not that difficult. There are examples all over the place of performing each part. The only thing you need to do is understand them, and incorporate the discrete pieces into your sketch.
I know how to send the server response to the serial monitor.
And I know how to write to the SD card using the SD library.
The problem is bringing them together, beacause as you already mentioned they can't be used simultaneously.
How do I buffer the data and how do I switch between Ethernet and SD. I couldn't find any examples for that part.
I will try to experiment a bit. Maybe you've got some sources?
How do I buffer the data
That depends to a large extent on how you are getting and displaying the data today. You could use either a String object or a char array.
The String object is easy, and manages memory allocation and where/how to append data for you. On the other hand, it is a resource hog.
how do I switch between Ethernet and SD.
Each uses a different chip select pin. The Ethernet uses pin 10. The SD card uses pin 4. Set one pin HIGH and other LOW to select a device. Swap to select the other device.
Set the pin for the device to use to LOW. This is explained here:
Thanks for the hints.
I am able to send data from a client PC to the arduino via HTTP Post now and save them to SD card.
My major problem now is the data tranfer rate. I can't get anywhere near what should be possible.
When I use a packet size of 16 bytes (packetBufSize=16) I need a delay of 40ms in the save routine to ensure that the whole file is transmitted.
Otherwise (lower delay time or larger packet size) the arduino only receives the first part of the file and claims that there's no more bytes available (client.available()=0).
I could not figure out why yet. Maybe someone can help?
Here is my code to receive the data:
if (httpPostStart) {
while (client.available() > 0) {
c = client.read();
buf[byteCount] = c;
++byteCount;
if (byteCount==packetBufSize) {
//PgmPrint("PACKET END\n");
save(buf);
byteCount = 0;
}
}
}
And here how it is stored to the SD card:
void save(char* data)
{
++saveCount;
useSD();
logfile.write(data);
delay(40);
if (saveCount==50) {
logfile.sync();
saveCount=0;
}
useEthernet();
}
I just can not figure out where the bottleneck is.
The data received from the client is transmitted with the speed of normal serial communication, 115200 baud and therefore 115200 bits/s. Am I right?
But when I try to receive large packets or decrease the delay time to increase transfer speed the ethernet connection seems to slow as data gets lost?
I can not get it faster than ~35 seconds for a 10kb file.
Has someone dealt with a similiar problem?
My whole sketch:
#include <SPI.h>
#include <Client.h>
#include <Ethernet.h>
#include <Server.h>
#include "SdFat.h"
#include "SdFatUtil.h"
//#include "WaveSD.h"
// HTTP TEXT
prog_char string_0[] PROGMEM = "HTTP/1.1 200 OK"; // "String 0" etc are strings to store - change to suit.
prog_char string_1[] PROGMEM = "Content-Type: text/html";
prog_char string_2[] PROGMEM = "<HTML><HEAD></HEAD>";
prog_char string_3[] PROGMEM = "<BODY>";
prog_char string_4[] PROGMEM = "<form method='post' enctype='multipart/form-data'>";
prog_char string_5[] PROGMEM = "<input type='file' name='data'><input type='submit'></form>";
prog_char string_6[] PROGMEM = "</BODY></HTML>";
PROGMEM const char *httpText[] =
{
string_0,
string_1,
string_2,
string_3,
string_4,
string_5,
string_6 };
#define packetBufSize 128
char buf[packetBufSize+1];
Sd2Card card;
SdVolume volume;
SdFile root;
SdFile logfile;
SdFile soundfile;
//WaveSD wave;
// store error strings in flash to save RAM
#define error(s) error_P(PSTR(s))
void error_P(const char* str) {
PgmPrint("error: ");
SerialPrintln_P(str);
if (card.errorCode()) {
PgmPrint("SD error: ");
Serial.print(card.errorCode(), HEX);
Serial.print(',');
Serial.println(card.errorData(), HEX);
}
while(1);
}
const int SDchipSelect = 8;
const int EthernetChipSelect = 10;
bool bPlayedSound = false;
bool httpPostDetected = false;
bool httpPostStart = false;
int byteCount = 0;
int saveCount = 0;
String received="";
String boundary="";
// Ethernet Configuration
Server server(80);
static uint8_t mac[] = {
0x90, 0xA2, 0xDA, 0x00, 0x22, 0x2F };
static uint8_t ip[] = {
192, 168, 178, 100};
static uint8_t gateway[] = {
192, 168, 178, 1};
//////////////////////////////////// SETUP
void setup(void) {
Serial.begin(115200);
pinMode(EthernetChipSelect, OUTPUT);
pinMode(SDchipSelect, OUTPUT);
useSD();
PgmPrint("Free RAM: ");
Serial.println(FreeRam());
// initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
// breadboards. use SPI_FULL_SPEED for better performance.
if (!card.init(SPI_FULL_SPEED, 8)) error("card.init failed");
// initialize a FAT volume
if (!volume.init(&card)) error("volume.init failed");
// open the root directory
if (!root.openRoot(&volume)) error("openRoot failed");
logfile.remove(&root, "playlog.txt");
useEthernet();
Ethernet.begin(mac, ip, gateway);
server.begin();
}
void loop(void)
{
useEthernet();
// listen for incoming clients
Client client = server.available();
if (client) {
// an http request ends with a blank line
boolean currentLineIsBlank = true;
while (client.connected()) {
if (client.available()) {
char c = client.read();
// if you've gotten to the end of the line (received a newline
// character) and the line is blank, the http request has ended,
// so you can send a reply
if (c == '\n' && currentLineIsBlank && !httpPostDetected) {
sendHtml(client, server);
break;
}
else if (c == '\n' && httpPostDetected && !httpPostStart) {
if (boundary == "") {
PgmPrint("TEST1\n");
Serial.print(received);
int search = received.indexOf("boundary=");
if (search!=-1)
{
boundary = received.substring(search+9);
PgmPrint("BOUNDARY READ\n");
}
}
else {
int search = received.indexOf(boundary);
if (search!=-1)
{
PgmPrint("BOUNDARY DETECTED\n");
PgmPrint("POST DATA START\n");
PgmPrint("AVAILABLE BYTES: ");
Serial.println(client.available());
delay(1);
useSD();
if (!logfile.open(&root, "playlog.txt", O_WRONLY | O_APPEND | O_CREAT))
error("can't open log file");
useEthernet();
httpPostStart = true;
}
}
received = "";
byteCount = 0;
}
else if (c == '\n' && !httpPostDetected) {
PgmPrint("NEW LINE\n");
// you're starting a new line
currentLineIsBlank = true;
int search = received.indexOf("POST");
if (search!=-1)
{
httpPostDetected = true;
PgmPrint("POST DETECTED\n");
}
received = "";
} /*
else if (c == '\n' && httpPostStart) {
currentLineIsBlank = true;
received.concat(c);
Serial.print(c);
if (httpPostStart) {
++byteCount;
if (byteCount==packetBufSize) {
PgmPrint("PACKET END\n");
save(received);
received = "";
byteCount = 0;
}
}
}*/
else if (c != '\r') {
// you've gotten a character on the current line
currentLineIsBlank = false;
received.concat(c);
Serial.print(c);
if (httpPostStart) {
while (client.available() > 0) {
c = client.read();
buf[byteCount] = c;
++byteCount;
if (byteCount==packetBufSize) {
//PgmPrint("PACKET END\n");
save(buf);
byteCount = 0;
}
}
}
}
}
else if (httpPostStart){
PgmPrint("Free RAM: ");
Serial.println(FreeRam());
useSD();
// IMPORTANT: close log file to flush any unwritten changes
logfile.close();
useEthernet();
PgmPrint("LAST POSTED DATA: ");
Serial.println(buf);
httpPostDetected = false;
httpPostStart = false;
boundary = "";
sendHtml(client, server);
break;
}
}
// give the web browser time to receive the data
delay(1);
// close the connection:
client.stop();
}
}
void sendHtml(Client client, Server server)
{
PgmPrint("SEND SERVER HTML");
char buffer[65]; // make sure this is large enough for the largest string it must hold
// send a standard http response header
for (int i = 0; i < 2; i++)
{
strcpy_P(buffer, (char*)pgm_read_word(&(httpText[i]))); // Necessary casts and dereferencing, just copy.
client.println( buffer );
}
client.println();
for (int i = 2; i < 7; i++)
{
strcpy_P(buffer, (char*)pgm_read_word(&(httpText[i]))); // Necessary casts and dereferencing, just copy.
server.print( buffer );
}
}
void save(char* data)
{
++saveCount;
useSD();
logfile.write(data);
delay(500);
if (saveCount==5) {
logfile.sync();
saveCount=0;
}
useEthernet();
}
void useEthernet()
{
digitalWrite(EthernetChipSelect, LOW);
digitalWrite(SDchipSelect, HIGH);
}
void useSD()
{
digitalWrite(SDchipSelect, LOW);
digitalWrite(EthernetChipSelect, HIGH);
}
frsc:
The data received from the client is transmitted with the speed of normal serial communication, 115200 baud and therefore 115200 bits/s. Am I right?
Yes, you're right, this is about 11520 bytes per second. This is the speed at which each byte is sent. What you don't account here for is the delay between characters, which can be anything and be caused by any component in your system. There are no easy solutions for this, you first need to understand where your program spends its time before even thinking about improving the performance. And while you're at it, don't forget that debug statements sent out on the serial take time too, so first check how your debug set-up affects the performance. Good luck.
Korman
Hey there, i am interested to know if you were able to get this "ethernet file transfer" to work? I am also trying to dynamically upload a wav file from the internet to an SD card for playback. In my case we are using a Wifly shield, microSD shield then with a wav shield for playback. Any advice or pointers that you may have would be much appreciated. i am happy to share final results too.
Thanks!