Go Down

Topic: Ethernet Shield sending byte by byte? (Read 6379 times) previous topic - next topic

myerika

When sending data (the response to the client) via the Ethernet Shield I can see in the Network Analyze program (Wireshark) all data are sent byte by byte. For that reason there are a lot of interactions between the client and the server. Are there any possibilities to send the response in one "block" over the network. Total data length is approx 300 bytes  

follower

That's a very interesting observation. In theory, yes, I think that should be possible. You'd need to look more closely to see how the library is implemented. Have you tried the Ethernet2 library? If so, does it function the same way?

--Phil.

myerika

Yes, I have tryed Ethernet2 and I got the same result, one byte sent from the server and then a response (Ack) from the client and then next byte ..... and so on.

follower

A quick look at this suggests the reason for the issue is that the client send code is implemented on top of the Print class and at the bottom of the "stack" is a routine that prints a byte. And to print a string that routine gets called multiple times.

I don't know if it's possible to override the higher-level methods but it would seem worthwhile as this must be adding a fair bit of overhead. The underlying ethernet code is certainly capable of sending multiple bytes.

--Phil.

follower

I took a look at this since I was using WireShark to track down another issue (with my code/Adobe Flash not the library :-)).

I can confirm what the OP posted, in my case for each byte of data sent via println 60 bytes were actually being sent on the wire--this includes for the response headers etc.

Unfortunately trying to solve this raises the whole issue of buffering etc which means it's not quite a straight fix of println. Although if the data isn't in PROGMEM then it's probably more straight-forward but that's hardly ideal.

Will ponder.

--Phil.

myerika

Reason for me to care about this is the communication cost. As you have seen there are a lot of overhead. I have the Arduino stuff in my boat and connected to a mobile network and I have to pay extra if I send to much data. Unfortunately I am not a programmer so I can't contribute with coding skills. I hope you can find a solution

Thomas in Stockholm    

follower

Hi,

I've taken a look at the code and have produced a modified copy of the Client library--as found in Arduino 0013--the new code provides better Arduino Ethernet transmission efficiency.

It's really only an interim solution and will only work if you make use of the println or print methods and supply a string.

I've only tested it with the standard WebServer example but that resulted in ~75% fewer bytes being transmitted. Even in this scenario there's still a lot of overhead as the numerals are still transmitted one byte at a time. Much more detail and analysis at the link above.

Please let me know how this works for you.

--Phil.

myerika

I can't get your code ?? Is your site down?

Thomas

follower

Quote
I can't get your code ?? Is your site down?

Unfortunately, it appears so. That was bad timing. :-/

Put this at the end of Client.cpp:
Code: [Select]

#if 1
#include <string.h>
void Client::print(const char c[]) {
 send(_sock, (uint8_t *) c, strlen(c));  
}

void Client::println(const char c[]) {
 print(c);
 print("\r\n");
}
#endif


And then this in the appropriate place (use the first and last lines to show you where) in Client.h:

Code: [Select]

 friend class Server;
#if 1
 void print(const char c[]);
 void println(const char c[]);
 using Print::print;
 using Print::println;
#endif
};


--Phil.

myerika

Very interesting.. number of sent/received bytes i exactly the same 444 in and 589 out from the Arduino. But number of interactions went down from 888 to 136, very good!!!

Thomas    ;D

follower

Great, thanks for the feedback.

--Phil.

myerika

Some more observations. As every PRINT and PRINTLN generate a "segment" "on the line" I have tryed to reduce the numbers of these statement and instead put in as many chars as possible per PRINT, about 10 to 20. But there seems to be some limitation in the code or the Arduino HW because I got some hang up in the communication at some point. Very strange, I reduce the number of PRINT but more data per PRINT up to some limit ???

Thomas    :-/

myerika

One more observation, when sending from a buffer like:
Char buf1[300]; and client.print(buf1); there seems to be a limit at 240-250 chars. I still use your new code Phil.

Thomas  

macegr

Additional information: apparently no one involved in developing the Ethernet Library has tested communications with web servers other than Apache. The library as-is will not communicate with lighttpd. Phil's modification above does allow communication, you will need to take care to render each line into a complete string before trying to println() it; multiple print() commands per line will not work.

So now I need to figure out why the Ethernet library hangs after four connection attempts at 5 second spacing, even with stop() command after each connection. At this point I know the connection idles in the FIN_WAIT state for less than a second up to 20 seconds. If another connection is attempted during this time, apparently the library makes a new one...it's limited to four. The Client::connected() function does not address this at all.
Unique RGB LED Modules and Arduino shields: http://www.macetech.com/store

macegr

All right, I think I have it figured out. If you try to connect again before the connection has completely ended, you will start another connection. This is fine as long as you don't go above four connections. The easiest way to not deal with connection pool management is to just wait until your current connection is really closed.

The act of checking the status register actually slows down the disconnect cycle. The code below should be placed after you have finished receiving all your data:
Code: [Select]
client.stop();
while(client.status() != 0) {
  delay(5);
}


Note that I'm using the status() function which is not exposed on the Ethernet library reference. This is a direct access to the W5100 Sn_SR status registers; all you need to know is that !client.connected() may not actually signify an ended connection. The delay(5) is necessary to give the W5100 some breathing room while it tries to terminate the connection.

Using this method, I am able to reliably download a web page about twice per second, probably a limit of the SPI and my processing routines. The time between downloads varies somewhat, but does not go over 1 second.

For ultimate reliability, you may want a timeout in the above code in case it somehow gets stuck waiting for a connection that never closes.
Unique RGB LED Modules and Arduino shields: http://www.macetech.com/store

Go Up