web server unexpected operation

Projects Networking, Protocols, and Devices
1

Hi,

I'm mostly a data acquistion guy and have leveraged the arduino forum to construct a web page to post the data. The web page has a fixed IP address. The sketch works great locally, however in full operation I get an occasional anomaly (connection reset from the browser), not sure if it is due to programming or the end application. Hoping for some guidance on how to determine the source of the error.

I'm using an arduino mega2560 from sunfounder with the older ethernet shield (R3) and measuring some temperatures and RPM of a turbine with a custom shield I have made. The system is located remotely in typical operation. The web page is served up through a couple ethernet switches and over a point to point wireless link and onto the internet via port forwarding using a global IP address. I get the same error on all the board stacks I have used, seems independant of the arduino stack hardware. I have a newer ethernet 101 board (for a trial) but have not yet integrated it to the system

Using arduino IDE serial monitor I capture the ethernet details from a browser data pull operation and they are included below. sort of mysterious to me.

//start serial capture
new client
GET / HTTP/1.1
Host: 192.168.1.18
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0

client disconnected
//end serial capture

The application works well locally, but when I pull the data remotely, I occasionally get a browser error that says the "connection reset" and all the browser data disappears. Same error dynamics using IE and firefox.

I note that in the sketch line 393 says to "disconnect client". I am wondering if the browser "connection reset" is due to this line of code or a failure in the remote link? as I said earlier, I never see this "connection reset" locally, only remotely. The remote data pull can take several seconds sometimes. I hope to increase the size of the data as a next step, so I need to solve this at this level of data capture.

Reduced sketch attached. this is a cut down version of my main application that makes the data capture generic, and has identical ethernet code for web page construction.

thanks in advance for your guidance,
tim
.

analog_shield_test_7_27_2016.ino (13.8 KB)

2

Sounds like you might have a routing problem. Does it work remotely now and then? It may be the speed and number of packets sent. Every time you call client.print, client.prinln, or client.write, it creates another packet.

3

I didn't expect each print or write would be a separate packet. We also have 3 cameras on the system and a weather station.

Regarding pacing, it does update a chunk at a time, sometimes the chunks are bigger than others. I thought the variability was due to other network traffic.

Would you have a suggestion on how to better optimize the scheme?

Thanks,

4

twilkers:
Hi,

I'm mostly a data acquistion guy and have leveraged the arduino forum to construct a web page to post the data. The web page has a fixed IP address. The sketch works great locally, however in full operation I get an occasional anomaly (connection reset from the browser), not sure if it is due to programming or the end application. Hoping for some guidance on how to determine the source of the error.

I'm using an arduino mega2560 from sunfounder with the older ethernet shield (R3) and measuring some temperatures and RPM of a turbine with a custom shield I have made. The system is located remotely in typical operation. The web page is served up through a couple ethernet switches and over a point to point wireless link and onto the internet via port forwarding using a global IP address. I get the same error on all the board stacks I have used, seems independant of the arduino stack hardware. I have a newer ethernet 101 board (for a trial) but have not yet integrated it to the system

Using arduino IDE serial monitor I capture the ethernet details from a browser data pull operation and they are included below. sort of mysterious to me.

//start serial capture
new client
GET / HTTP/1.1
Host: 192.168.1.18
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0

client disconnected
//end serial capture

The application works well locally, but when I pull the data remotely, I occasionally get a browser error that says the "connection reset" and all the browser data disappears. Same error dynamics using IE and firefox.

I note that in the sketch line 393 says to "disconnect client". I am wondering if the browser "connection reset" is due to this line of code or a failure in the remote link? as I said earlier, I never see this "connection reset" locally, only remotely. The remote data pull can take several seconds sometimes. I hope to increase the size of the data as a next step, so I need to solve this at this level of data capture.

Reduced sketch attached. this is a cut down version of my main application that makes the data capture generic, and has identical ethernet code for web page construction.

thanks in advance for your guidance,
tim
.

I am surprised that it works as well as it does.

I just ran a compile on your example:

Sketch uses 16,632 bytes (6%) of program storage space. Maximum is 253,952 bytes.
Global variables use 8,002 bytes (97%) of dynamic memory, leaving 190 bytes for local variables. Maximum is 8,192 bytes.
Low memory available, stability problems may occur.

Your example only has 190 bytes of unallocated RAM.

How deep is your deepest function call? how many local variables are allocated during the deepest call?

I rewrote your code, I used ProgMem to increase your free RAM, removed duplicate loops by creating a function.

Sketch uses 15,330 bytes (6%) of program storage space. Maximum is 253,952 bytes.
Global variables use 7,743 bytes (94%) of dynamic memory, leaving 449 bytes for local variables. Maximum is 8,192 bytes.
Low memory available, stability problems may occur.

It compiles, but, I don't know if it actually works. :slight_smile:

Chuck.

analog_shield_test_7_27_2016_1.ino (16.8 KB)

5

If it is working ok locally, and has problems on the internet, it may be routing. Try a very simple version of the server response to see if it works any better.

6

twilkers:
I didn't expect each print or write would be a separate packet. We also have 3 cameras on the system and a weather station.

Regarding pacing, it does update a chunk at a time, sometimes the chunks are bigger than others. I thought the variability was due to other network traffic.

Would you have a suggestion on how to better optimize the scheme?

Thanks,

in your loops where you are writing one decimal value with a comma and space separator, try this:

#define BUFFERLEN=256;

char buf[BUFFERLEN]; // if you have enough  free ram?
int b=0;
for(int a=0;a<max;a++){
  b+=sprintf_P((char*)buf[b],PSTR("%d, "),sample[a]);

  if (b>(BUFFERLEN-10)){ // buffer is almost full so print it
    // each time through the loop, the maximum added to buffer should only be 5+2+1,(5 digits, comma,space, and the \0 string terminator
    client.print(buf);
    b=0; // reset buffer point back to beginning
    }
  }
if(b!=0){ // somthing left over in buffer print it
  client.print(buf);
  b=0;
  }
}

Chuck.

7

Chuck, works fine. need some fine tuning on single variables, but this is a minor issue. thanks! it will take me a bit of time to work this into my remote application as a point of reference.

8

having trouble printf a float. does anyone know a better way to do this than below?
http://forum.arduino.cc/index.php?topic=44262.0
thx

9

twilkers:
having trouble printf a float. does anyone know a better way to do this than below?
How to sprintf a float? - Syntax & Programs - Arduino Forum
thx

Sorry about that, I forgot that the Arduino does not support floats in printf statements. It was decided to exclude support for floats due to code bloat in the format routines.

Here is a solution from StackOverFlow Arduino Float to Char

char str_temp[6];

/* 4 is mininum width, 2 is precision; float value is copied onto str_temp*/
dtostrf(temp, 4, 2, str_temp);
sprintf(temperature,"%s F", str_temp);

If you really want to get tricky, dtostrf returns a pointer to its destination string, so this should work:

char buf[60];

uint8_t b=sprintf_P(buf,PSTR("this is a big float %s"),dtostrf(Temp,9,6,(char*)buf[50]));

// this assumes that buf[50..59] are not being used already.  In this explicit case, only the 
// first 20 character are in use before the 'string' is appended.

Chuck.

10

I am so confused.

I have this working in another sketch, now porting to the real end application. I call the function
print_samples() on line 307 but it is never entered and no error from compiler.

I moved the definition all around
a. between setup() and loop()
b. at the end of loop()

never actually gets called.

cut down storage a lot.
Sketch uses 20,576 bytes (8%) of program storage space. Maximum is 253,952 bytes.
Global variables use 1,869 bytes (22%) of dynamic memory, leaving 6,323 bytes for local variables. Maximum is 8,192 bytes.

still never actually gets called.

what is keeping the function from getting executed?

thanks,
tim

temp_sketch_mega_8_11_2016_with_optimized_packets_and_RPM.ino.ino (16.4 KB)

11

twilkers:
I am so confused.

I have this working in another sketch, now porting to the real end application. I call the function
print_samples() on line 307 but it is never entered and no error from compiler.

I moved the definition all around
a. between setup() and loop()
b. at the end of loop()

never actually gets called.

cut down storage a lot.
Sketch uses 20,576 bytes (8%) of program storage space. Maximum is 253,952 bytes.
Global variables use 1,869 bytes (22%) of dynamic memory, leaving 6,323 bytes for local variables. Maximum is 8,192 bytes.

still never actually gets called.

what is keeping the function from getting executed?

thanks,
tim

Tim,
Remove the local declaration of client() on line 258, You already have a global declaration on line 98.

When you execute the code on line 258

// listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");

You are creating a NEW object client() Then the call to print_samples() uses the Global client() that has never been initialized, it goes crazy.

Change it to this:

// listen for incoming clients
  client = server.available();
  if (client) {
    Serial.println("new client");

I deleted the EthernetClient on line 258, and changed the IP address to match my network. Attached is a screen shot

Chuck.

screen.jpg
screen.jpg913×1014 162 KB

12

Thanks Chuck!