I've build a datalogger with mega2560 and ethernetshield/sd card.
It logs a file 1 MB large. When I get them from a browser from my pc from the arduino, it sends about 80 MB.
I send the data char based.
code:
void readDataCmd(WebServer &server, WebServer::ConnectionType type, char url_tail, bool tail_complete)
{
server.httpSuccess();
device("SD");
snprintf(msg,sizeof msg, "Reading file ... %s for http readdata request",data_file); writeLog(msg);
if ( !(file.open(&root, data_file, O_READ))) { snprintf(msg,sizeof msg, "error reading config file "); writeError(msg);}
int16_t n; uint8_t buf[100];// nothing special about 7, just a lucky number.
while ((n = file.read(buf, sizeof(buf))) > 0)
{
device("Ethernet");
for (uint8_t i = 0; i < n; i++) server.print(buf);*
device("SD");*
*} * file.close(); //Close the file snprintf(msg,sizeof msg,"Closing config file ... "); writeLog(msg); } 1. is it normal that 1MB transfert took 80 MB data sending over network ? 2. Does anyone know a good solution to get files (till 10MB) from the arduino over ethernet to the pc in a quick way (other then char per char) ? Thanks, Jeroen.
If you failed to open the file, you print a message. Do you then return? No. You proceed to try to read from the file, anyway.
while ((n = file.read(buf, sizeof(buf))) > 0)
{
device("Ethernet");
for (uint8_t i = 0; i < n; i++) server.print(buf[i]);
device("SD");
}
In this function, you just keep reading from the file. Does the file.read function actually check end of file? Or does it just get the next n bytes from the card.
You are relying on a full blown file system on the SD card/library, whereas that is not what you actually have.
If you are logging data, presumably you know how much data you have logged. When sending the data to the client, don't send any more data than you have logged.
Read data from a file starting at the current position.
\param[out] buf Pointer to the location that will receive the data.
\param[in] nbyte Maximum number of bytes to read.
\return For success read() returns the number of bytes read.
A value less than \a nbyte, including zero, will be returned
if end of file is reached.
If an error occurs, read() returns -1. Possible errors include
read() called before a file has been opened, corrupt file system
or an I/O error occurred.
*/
So What I understand is that when it reaches and of file, it will give a value back less than the amount of buf (in my case 100). When you read again, it will give 0. So the end of file is tested ?
For not confusing the Russians: So, my logfile on arduino sd is 1,4 MB, when I receive it via pc, it is 1,4 MB on my pc, but I see 80 MB of tranfert needed for this file. Are you sure this is not correct (sending byte by byte, and for every byte a tcp-ip pakcet of 4 kb ?)
It also takes about 10 min to transfert.
OK, so yes, it appears as though end of file is detected.
Have you compared the first 1.4M bytes of the file on the SD card to the first 1.4M bytes of the received file? Do they match?
The server.print() function inherits from the same base class as Serial.print(), so it should take all the same arguments, mandatory and optional.
It could be that you need to add ",BYTE" to the server.print() call.
One thing you might try is to make the buffer smaller, say 56 bytes, then print the whole buffer, not each individual byte. Depending on how the server is configured, each byte could result in a packet being sent. Printing the whole buffer, assuming it fits in a packet (which 56 bytes should), might result in less traffic and less overhead.
I would think that the ethernet chip would fill packets itself, but perhaps this isn't happening.
Yes, the received file (on pc) is exactly the same as the original on sd.
Adding the ,BYTE on the server.print statement, I can't find this option in the documentation (Ethernet - Arduino Reference).
What exactly does it mean: sending the char as byte ?
I will try to lower the buffer to 32 or 56 and try to print the hole buffer at once and look if the transfert will be faster.
The Server class derives from the Print class. Being a base class, it does not appear that the Print class IS documented. I guess it is up to the derived classes to document it.
The Serial class also derives from Print, and it does a better job of documenting the print method.
The best, most up-to-date documentation is the class itself, though.
What exactly does it mean: sending the char as byte ?
Yes, as opposed to converting the value to a string. Since the buffer type is not char (even though it is the same size), I thought that your problem was that you were not receiving the file correctly, so I suggested this in an attempt to remedy the problem.
Now, I see that you are receiving the file correctly, but that the signal to noise ratio is too high.
I think that the problem then is that each server.print is sending a packet. You need to include more data in each print, to reduce the number of prints and packets.