Show Posts
Pages: [1] 2 3 ... 10
1  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet Shield Unreliable? on: January 01, 2012, 09:26:54 pm
This is problematic:
Code:
client[connection].println("GET " + request);
    client[connection].println();
   

It's practically a bug in the ethernet library, but client.print and println shouldn't be used with strings, it causes the strings to be sent out one byte at a time which can overflow the send buffer in the ethernet shield.  You need to use client.write(), which means you need to use something other than a string.
2  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet Shield Unreliable? on: December 31, 2011, 11:14:48 pm
Three things I have found:

1. If you have the ethernet shield that also has the SD, you need to disable the SD or it can interfere with the ethernet.  You do this by putting these lines in your setup routine:
    pinMode(4,OUTPUT);
    digitalWrite(4,HIGH);

2. The ethernet card can hang if you overwhelm it with writes. If you use Ethernet::print() it can sometimes trigger one write operation per byte. Using Ethernet::write() avoids this, causing one write for each write statement.

3. The LED pin (13) on an UNO is also used by the ethernet card and must not be used.
3  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 24, 2011, 10:06:53 pm
So in changing all of my code from client.print to client.write my code compiles, but when I run it, it hangs on this line of code:

Code:
sprintf(servBuffB,"<font size=32>Setpoint Fermenter %d : <input type=text style=font-size:32pt size=3 name=%c value=%d /> Open</font><br />", j + 1, Whereto[j], setferm[j]);

Where j is anything from 0 to 3 and :
Code:
int setferm[4] = {300,300,300,300};
String Whereto[4] = {"a","b","c","d"};

I narrowed it down to the "%c" being the reason it is crashing.  I've also tried %s and %d with no luck....  I'm guessing my syntax is messed up somehow?  I have other lines with just the %d in there at it works fine, but it realldoesn't seem to like my Whereto[] array. Any help is highly appreciated.

Try:
char Whereto[] = {'a','b','c','d'};

and %c in the format string.
4  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 24, 2011, 10:35:28 am
An update --

I modified my program so that:
* I pull pin 4 high in setup()
* I no longer blink the LED on pin 13
* I no longer call ethernet.begin() before each GET
* I replaced all of my calls to client::print() with a single call to client::write() using a buffer that I assemble

Of these, I think that the last one is the significant one. I think the others are good practice but not the big difference.

The results:
On the first iteration with the new modifications I have so far done 3038 GET's without an error. In previous versions of the sketch my all-time record was 1553 GET's. What my sketch is doing is contacting a web server on the internet every 20 seconds and running a php script.

What is clear to me is that client::print() is poorly implemented and should be avoided. Using it tends to create a lot of socket writes, which exposes issues in the underlying layers. Where those issues are I don't know. Using client::write(), which maps one-to-one with socket writes, is a better choice and seems much less likely to expose underlying issues.


As a final test, after my program had successfully done 8493 consecutive GETs, I changed it to use Client::print(String) instead of Client::write(char *). This is the most pathological version of print(), which creates one socket send for each byte in the string. The results:

So far, 2159 GETs without an error.

Not at all what I was expecting.

Maybe pin 13 and pin 4 are more important than I thought...
5  Using Arduino / Networking, Protocols, and Devices / Re: W5100 for $20? on: October 24, 2011, 10:17:57 am
The last ethernet shield I bought was $29 including shipping on ebay. It hardly seems worthwhile to mess around to save $9.
6  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 22, 2011, 02:42:31 pm
Whenever you write a network application, you have to prepare for the possibility that your application will generate data faster than the network can take it. The application author then has two choices: discard data or wait.  If timeliness is most important, say in streaming video, then your only choice is to discard some of the data. If data completeness is most important then you have to wait.

So a network library should always give the developer the choice of waiting (by blocking)  or not (by returning failure if the buffer is full). Usually a compromise is also offered, where blocking will fail if a time-out is exceeded. Beyond that a network library should just be a black box at the application level, put data in one end and it comes out the other.

I guess what I'm saying is the Arduino ethernet library needs some work at the conceptual level.
7  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 22, 2011, 02:28:31 pm
An update --

I modified my program so that:
* I pull pin 4 high in setup()
* I no longer blink the LED on pin 13
* I no longer call ethernet.begin() before each GET
* I replaced all of my calls to client::print() with a single call to client::write() using a buffer that I assemble

Of these, I think that the last one is the significant one. I think the others are good practice but not the big difference.

The results:
On the first iteration with the new modifications I have so far done 3038 GET's without an error. In previous versions of the sketch my all-time record was 1553 GET's. What my sketch is doing is contacting a web server on the internet every 20 seconds and running a php script.

What is clear to me is that client::print() is poorly implemented and should be avoided. Using it tends to create a lot of socket writes, which exposes issues in the underlying layers. Where those issues are I don't know. Using client::write(), which maps one-to-one with socket writes, is a better choice and seems much less likely to expose underlying issues.
8  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 20, 2011, 03:04:01 pm
Quote
Yes, I've seen that.

Any sudden changes in TX_WR are an illusion caused by interference with the SD card. If you're still seeing them then you didn't fix the SD card SPI problem.

There is no longer any interference with the SD card. Not a single fail in thousands of downloads now.



Just so we're all on the same page, is this a correct summary of your current status:
1. Pin 4 set to high to disable SD card
2. "Correction Code" that monitors for changes in TX_WR
3. Zero failures in that configuration so far
?

Is it also true that both (1) and (2) are necessary and that if either is omitted there will be failure?
9  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 20, 2011, 12:18:56 pm
@DCContrarian: That is not correct. Those registers are in the w5100, not the Arduino. I am reading them over the SPI connection. That jumping around is happening in the w5100.


You're right,  readSnTX_WR(s) and readSnTX_RD(s) send an SPI message to the w5100 asking for the state of those registers.  I didn't dig enough.
10  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 20, 2011, 11:21:12 am
Did some more testing

client.print("Content-Length:");  will send a single packet of length 15

char* stuff = {"Content-Length:"};
client.print(stuff};                       will send a single packet of length 15

strcpy_P(Dbuf,PSTR("Content-Length:"));  will send a single packet of length 15
pachube.print(Dbuf);

Whereas,

int stuff = pick a number;
client.print(stuff, DEC);                     will send several packets 1 byte in length

So, it seems that formatting variables for display is where the problem could happen, otherwise it's no big deal.  Therefore, don't use client.print for sending individual variables.  Format them into a larger message like the sprintf examples we've shown and in general, the less prints, printlns, writes, you do, the less packets you'll be sending.  That makes sense, but is a far cry from the warnings I've seen about every single character being put into a separate packet when one uses print.

We should watch that, it will really drive the new folks nuts.

But:
String stuff = {"Content-Length:"};
client.print(stuff};                      

will result in 15 sends of length one.

It will go through Print::print(), which looks like this:

Code:
void Print::print(const String &s)
{
  for (int i = 0; i < s.length(); i++) {
    write(s[i]);
  }
}

That's where the danger is.
11  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 20, 2011, 11:16:29 am


If the w5100 had some manners and had learned to share, it would know not to change the TX_WR register. That belongs to the Arduino.



At the risk of being pedantic, I'll reiterate that the W5100 has no ability to write to the Arduino memory or registers. It just has a simple serial connection using SPI.

When SPI is started, a timer interrupt is enabled on the Arduino. A few thousand times a second that interrupt goes off, and when it does the Arduino calls an interrupt service routine which checks the SPI registers to see if a read or a write is pending. If one is, the Arduino writes to the MOSI line, strobes the clock, reads the MISO line, and updates the registers.

The leaping around of registers you see is being done by the Arduino in the interrupt service routine. 

Note that the registers and buffers used by SPI are just part of the Arduino memory and could be overwritten by a buffer overflow or a pointer error in a sketch.

I have no idea why the transmit buffer doesn't behave like a simple ring buffer, or whether that is a feature or a bug.
12  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 19, 2011, 09:46:19 pm
I do exactly the same thing you do, except I use print.  Your example would be:

outBuf[128];
sprintf(outBuf,"Results testA: %u testB: %u testC: %u testD: %u\r\n",testA,testB,testC,testD);
clent.print(outBuf);

Based on my tests, we devolve down to exactly the same number of sends.  So, six of one, half dozen of the other.  To be truthful though, I often resort to client.println();  to just get a carriage return out there.  Lazy, I know, what can I say.

Print() calls Client::write(). Each call to Client::write() results in one call to send() which results in one command being sent to the w5100.

Where it gets tricky is that print has 9 overrides, and only one of them -- print(const char str[]) -- results in a single call to write().  All of the other overrides -- including the seemingly similar print(const String &s) -- result in one command per byte sent.  So it is possible to have print() work just as efficiently as write() -- but only in the case where you're not using any of the functionality of print(). The risk of using print() is that it's easy for an override to not work out the way you thought it might.  So there is no advantage to using print(), but there is a downside.
13  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 19, 2011, 01:36:05 pm

Edit2: This "correction code" undo has me a bit concerned. The ethernet shield is still violating one of my prime directives of circular queues. It changes the TX_WR register. That should be only done by the Arduino code.

This was the code that prevented another write until the w5100 was finished sending everything. So if the w5100 is using this pointer to build packets, the worst case scenario would be to use client.print() or client.println() to send data. That should cause that failure quickest, because all characters are sent in their own packet. The chances of failure due to this error should be greatest in that condition. I use client.write(). It sends the request in one packet. So I send a couple packets every 20 seconds. If you use the client.print(), it sends about 30 packets for the same request.


I want to clarify something here. The connection between the shield and the Arduino is SPI, which is a serial interface. The shield has no access to the memory or registers of the Arduino and doesn't write to or change anything.   (See http://en.wikipedia.org/wiki/Serial_Peripheral_Interface_Bus for details).

It is the Arduino SPI library that is changing things behind the scenes. This is interrupt-driven, so it is going on in the background of your sketch. If there is unpredictable behavior it is because of some interaction between the Ethernet and SPI libraries.
14  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 19, 2011, 01:14:16 pm
That would be the highest risk of the failure if this is the case. Each character is sent in its own packet. That request is gonna send a bunch of packets in a hurry. Then it waits until the next send, then sends a bunch of packets in a hurry.

The more packets you build together back-to-back like that, the better chance you have of this type of data collision.


Just to expand a little more on what SurferTim is saying here: the designers of the ethernet library decided to derive the Client class from Print and Stream so that it could use Print() and PrintLn() for formatted output. The problem with that is that Print() ends up calling write(byte) once per byte output, which causes one socket send for each byte sent. It's possible that the W5100 does some sort of defragmentation internally, but even so this is a poor way of implementing it.  One solution would have been to implement some sort of buffering in Client::write(), but that is non-trivial, consumes memory and increases latency.

Failing a re-write of Client:write(), it seems that it would be imprudent to use Client::print() or printLn().  Rather, you should format a buffer yourself and use either Client::write(const char *str) or Client::write(const uint8_t *buf, size_t size) instead.
15  Using Arduino / Networking, Protocols, and Devices / Re: Ethernet (w5100) sketch hangs on: October 18, 2011, 04:40:56 pm
The soft reset will only work some of the time.  smiley-sad


Note that a soft reset -- at least the only way I know how to do it -- isn't really a reset, it's a restart. It's jumping back to 0, the program entry point. If things have gotten screwed up that doesn't unscrew them.
Pages: [1] 2 3 ... 10