Show Posts
Pages: [1] 2
1  Using Arduino / Networking, Protocols, and Devices / Re: Web client + NTS reboots system on: April 29, 2013, 06:37:19 pm
Well, bumping up pageAdd seems to help. Strange because the obscured url is in reality only 26 characters.
Also strange that the sketch worked for hours without the NTP code. Or maybe not, because overrunning the allocated space for pageAdd simply ended up in unused space with no code claiming this space. Or something like that.
Of course the 5 sec interval was only to test if it was indeed the start of NTP that caused the reset. Like I said, in real life the interval is set to 6 hours or longer. As you will have noticed, I also use pool.ntp.org to lessen the strain on one particular server.

Thanks for your suggestion, I'll leave the system running for a while to see if it is stable.
2  Using Arduino / Networking, Protocols, and Devices / Web client + NTS reboots system on: April 29, 2013, 06:10:51 pm
I was working on a sketch where my Arduino is both a server, a web client and reads time from a NTP-server. It worked, but only for a while. After a couple of hours either the server stopped responding or the client did not get the data. I searched the forums and found some interesting stuff from SurferTim where he implements a timeout-function in case something goes wrong. So I started with the web-client from the playground, added my own addresses and let it get data (a simple xml file containing some setup data) from my (Apache) server for a couple of hours without crashes. Then I added the NTP code from my other sketch. This works fine on startup, but once in loop the Arduino resets itself every time the NTP code is called. This did not happen in my other (much larger) sketch. I understand that auto-rebooting is usually caused by a buffer overflow or memory shortage. So I also added code to check the amount of free memory. This reports 1280 bytes free and never changes.
When I out-comment the 'getPage' line from loop I get a new NTP-IP every 5 seconds. So the NTP code on its own works, the client code also works but together they cause havoc.
Note that I poll an NTP server very 5 seconds only for testing. In my first try I had not set this interval which means it polls every 5 minutes. The Arduino would reboot after 86 loops (with the loop set to 3 seconds that's about 5 minutes, also the syncInterval time when not set with setSyncInterval() if you call now() often enough). I set it to 5 seconds to check if polling the NTP server was what caused the crash. In real life my sketch can do with a poll once every 6 hours or longer.
Also note I xxx-ed out my server address. Also not I use port 6960 for my client. That is not a mistake, my Apache server is configured to also listen on that port.

Code:
/*
   Web client sketch for IDE v1.0.1 and w5100/w5200
   Uses GET method.
   Posted October 2012 by SurferTim
*/

#include <Time.h>
#include <SPI.h>
#include <Ethernet.h>
#include <EthernetUdp.h>
#include <Dns.h>


// this must be unique
byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

// change to your network settings
IPAddress ip(192,168,0,177);
IPAddress dnsIP(8,8,8,8); // google dns
IPAddress gateway(192, 168, 0, 1);
IPAddress subnet(255, 255, 255, 0);
IPAddress server(1xx,1xx,xx,1xx); // server IP-number

// change to your server's port
int serverPort = 6960;

EthernetClient client;
int totalCount = 0;
int loopCount = 0;
char pageAdd[32];

/* Timeserver setups */

IPAddress timeServer; //
EthernetUDP Udp;
unsigned int localPort = 8888;  // local port to listen for UDP packets
const int timeZone = 2;     // Central European Time


void setup() {
  Serial.begin(9600);

  // disable SD SPI
  pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

  // Start ethernet
  Serial.println(F("Starting ethernet..."));
  Ethernet.begin(mac, ip, dnsIP, gateway, subnet);

  Serial.println(Ethernet.localIP());

  delay(2000);
 
  setSyncProvider(getNtpTime);
   
  setSyncInterval(5);
   
  //digitalClockDisplay();

  Serial.println(F("Ready"));
}

void loop()
{
  if(loopCount < 4)
  {
    // if loopCount is less than 30, just delay a second
    delay(1000);
  }
  else
  {
    // every thirty seconds this runs
    loopCount = 0;

    // Modify next line to load different page
    // or pass values to server
    //sprintf(pageAdd,"/",totalCount);

    sprintf(pageAdd,"http://www.xxx.xxx/xxx.xml",totalCount);

    if(!getPage(server,serverPort,pageAdd)) Serial.print(F("Fail "));
    else Serial.print(F("Pass "));
    totalCount++;
    Serial.println(totalCount,DEC);
    printRAM();
  }   
  loopCount++;
}

byte getPage(IPAddress ipBuf,int thisPort, char *page)
{
  int inChar;
  char outBuf[128];

  Serial.print(F("connecting..."));

  if(client.connect(ipBuf,thisPort))
  {
    Serial.println(F("connected"));

    sprintf(outBuf,"GET %s HTTP/1.0\r\n\r\n",page);
    client.write(outBuf);
  }
  else
  {
    Serial.println(F("failed"));
    return 0;
  }

  // connectLoop controls the hardware fail timeout
  int connectLoop = 0;

  while(client.connected())
  {
    while(client.available())
    {
      inChar = client.read();
      Serial.write(inChar);
      // set connectLoop to zero if a packet arrives
      connectLoop = 0;
    }

    connectLoop++;

    // if more than 10000 milliseconds since the last packet
    if(connectLoop > 10000)
    {
      // then close the connection from this end.
      Serial.println();
      Serial.println(F("Timeout"));
      client.stop();
    }
    // this is a delay for the connectLoop timing
    delay(1);
  }

  Serial.println();

  Serial.println(F("disconnecting."));
  // close client end
  client.stop();

  return 1;
}


/*-------- NTP code ----------*/

const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message
byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming & outgoing packets

time_t getNtpTime()
{
    Udp.begin(localPort);
   
    DNSClient dns;
    dns.begin(Ethernet.dnsServerIP());
   
    if(dns.getHostByName("pool.ntp.org",timeServer)) {
        Serial.print(F("NTP server ip: "));
        Serial.println(timeServer);
    }
    else Serial.print(F("dns lookup failed"));
   
    while (Udp.parsePacket() > 0) ; // discard any previously received packets
    //Serial.println("Transmit NTP Request");
    sendNTPpacket(timeServer);
    uint32_t beginWait = millis();
    while (millis() - beginWait < 1500) {
        int size = Udp.parsePacket();
        if (size >= NTP_PACKET_SIZE) {
            //Serial.println("Receive NTP Response");
            Udp.read(packetBuffer, NTP_PACKET_SIZE);  // read packet into the buffer
            unsigned long secsSince1900;
            // convert four bytes starting at location 40 to a long integer
            secsSince1900 =  (unsigned long)packetBuffer[40] << 24;
            secsSince1900 |= (unsigned long)packetBuffer[41] << 16;
            secsSince1900 |= (unsigned long)packetBuffer[42] << 8;
            secsSince1900 |= (unsigned long)packetBuffer[43];
            return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR;
        }
    }
    Serial.println("No NTP Response :-(");
    return 0; // return 0 if unable to get the time
}

// send an NTP request to the time server at the given address
void sendNTPpacket(IPAddress &address)
{
    // set all bytes in the buffer to 0
    memset(packetBuffer, 0, NTP_PACKET_SIZE);
    // Initialize values needed to form NTP request
    // (see URL above for details on the packets)
    packetBuffer[0] = 0b11100011;   // LI, Version, Mode
    packetBuffer[1] = 0;     // Stratum, or type of clock
    packetBuffer[2] = 6;     // Polling Interval
    packetBuffer[3] = 0xEC;  // Peer Clock Precision
    // 8 bytes of zero for Root Delay & Root Dispersion
    packetBuffer[12]  = 49;
    packetBuffer[13]  = 0x4E;
    packetBuffer[14]  = 49;
    packetBuffer[15]  = 52;
    // all NTP fields have been given values, now
    // you can send a packet requesting a timestamp:
    Udp.beginPacket(address, 123); //NTP requests are to port 123
    Udp.write(packetBuffer, NTP_PACKET_SIZE);
    Udp.endPacket();
}
void digitalClockDisplay(){
    // digital clock display of the time
    Serial.print(hour());
    printDigits(minute());
    printDigits(second());
    Serial.print(" ");
    Serial.print(day());
    Serial.print(" ");
    Serial.print(month());
    Serial.print(" ");
    Serial.print(year());
    Serial.println();
}

void printDigits(int digits){
    // utility for digital clock display: prints preceding colon and leading 0
    Serial.print(":");
    if(digits < 10)
        Serial.print('0');
    Serial.print(digits);
}

void printRAM()
{
    Serial.print("Free RAM = ");
    Serial.println(freeRam());
    digitalClockDisplay();
}

int freeRam () {
    extern int __heap_start, *__brkval;
    int v;
    return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

What could be wrong with this code?
3  Using Arduino / Networking, Protocols, and Devices / Re: Webserver & external files from SD card, no show... on: April 23, 2013, 01:37:29 pm
Slightly different approach, learned something again so thanks!

A
4  Using Arduino / Networking, Protocols, and Devices / Re: Webserver & external files from SD card, no show... on: April 21, 2013, 05:41:28 pm
OK, that also explains why the code I copied probably worked in it's original form. It had an 'extract filename' function:

Code:
void ExtractFileName()
{
  int i, j;
 
  i=0;
  while (header[i++] != ' ') {
    ;
  }
  j=0;
  while (header[i] != ' ') {
    filename[j++] = header[i++];
  }
  filename[j] = 0;
  if (filename[0] == '/' && filename[1] == 0) {
    strcpy(filename, "index.htm");
  }
}

And in stead of just opening SD.open("index.htm") it had:

Code:
                  ExtractFileName();
                  Serial.print("Extracted filename is: ");
                  Serial.println(filename);
                  webFile = SD.open(filename);            // open web file


However I kept getting "Extracted filename is /index.htm"  "file not found".
I now put back that code and now I see the picture appear. As far as I know the code now is identical to what I was doing before (giving me the constant "file not found") so I don't quite understand what went wrong the first time. As it turns out it's a good thing though.
Your explanation put me back on track, I now (sort of) understand what this code is doing and why it's there. Time to dive deeper into this.

Thanks a lot,

A
5  Using Arduino / Networking, Protocols, and Devices / Re: Webserver & external files from SD card, no show... on: April 21, 2013, 04:51:10 pm
Thanks, that makes perfect sense. But even though it's not rocket science to figure out what to do, figuring out how to do it is beyond me at the moment. (As is the reason this code appears to work for others, but that's another topic).
I've looked for tutorials and examples for hosting files from a SD card, but so far I've not found much (except for the forementioned examples that don't seem to work for me). I don't want anybody to write the code for me, but maybe a pointer to a tutorial dealing with this problem?
6  Using Arduino / Networking, Protocols, and Devices / Webserver & external files from SD card, no show... on: April 21, 2013, 04:18:03 pm
HI,

I'm experimenting with a webserver. Everything (UNO, 5100 Ethernetshield) runs fine as long as I put all the html code inline in the sketch.
When I try to load the html from a SD card only the text shows. I shamelessly copied the following sketch from another post on this forum (http://arduino.cc/forum/index.php/topic,158718.0.html), I only reduced the buffer sizes to 64.
I tried other examples too, i.e. the one from this post: http://arduino.cc/forum/index.php?topic=156244.0 and other from around the web, but every time only text shows up. No javascript loads, no external css files load etc.

This is the code I tried before giving in and ask for help:

Code:
#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

// MAC address from Ethernet shield sticker under board
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0xE7, 0xAC };
IPAddress ip(192, 168, 16, 100); // IP address, may need to change depending on network
byte dn[] = {8, 8, 8, 8 };
byte gw[] = { 192, 168, 16, 254 };
byte mask[] = { 255, 255, 255, 0 };
EthernetServer server(80);  // create a server at port 80

File webFile;
char header[64];
char filename[64];
int txtlen;
uint8_t buf[64];
int aantal;

void setup()
{
    pinMode(10, OUTPUT);
    Ethernet.begin(mac, ip, dn, gw, mask);
    server.begin();           // start to listen for clients
    Serial.begin(9600);       // for debugging
   
    // initialize SD card
    Serial.println("Initializing SD card...");
    if (!SD.begin(4)) {
        Serial.println("ERROR - SD card initialization failed!");
        return;    // init failed
    }
    Serial.println("SUCCESS - SD card initialized.");
}

void loop()
{
    EthernetClient client = server.available();  // try to get client

    txtlen = 0;
    if (client) {  // got client?
        boolean currentLineIsBlank = true;
        while (client.connected()) {
            if (client.available()) {   // client data available to read
                char c = client.read(); // read 1 byte (character) from client
                if (txtlen < 63) {
                  header[txtlen++] = c;
                  header[txtlen] = 0;
                }
                // last line of client request is blank and ends with \n
                // respond to client only after last line received
                Serial.write(c);
                if (c == '\n' && currentLineIsBlank) {
                  webFile = SD.open("index.htm");            // open web file
                  if (!webFile) {
                    Serial.println("File not found!!");
                    client.println("HTTP/1.1 404 NOT FOUND");
                  }
                  else {
                    // send a standard http response header
                    client.println("HTTP/1.1 200 OK");
                    client.println("Content-Type: text/html");
                  }
  client.println("Connection: close");
                  client.println();
                    // send web page
                  if (webFile) {
                        while(aantal = webFile.available()) {
                            if (aantal > 63) {
                              webFile.read(buf, 64);
                              client.write(buf, 64);
                            }
                            else {
                              webFile.read(buf, aantal);                             
                              client.write(buf, aantal); // send web page to client
                            }
                        }
                        webFile.close();
                  }
                  break;
                }
                // every line of text received from the client ends with \r\n
                if (c == '\n') {
                    // last character on line of received text
                    // starting new line with next character read
                    currentLineIsBlank = true;
                }
                else if (c != '\r') {
                    // a text character was received from client
                    currentLineIsBlank = false;
                }
            } // end if (client.available())
        } // end while (client.connected())
        delay(100);      // give the web browser time to receive the data
        client.stop(); // close the connection
    } // end if (client)
}

and this is the html:

Code:
<html>

<head>
<meta http-equiv="Content-Language" content="nl">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<title>This picture is from the Arduino</title>
</head>

<body>

<p><font size="5">This picture is from the Arduino webserver:</font></p>
<p>
<img border="0" src="picture.jpg" width="349" height="262"></p>
<p>&nbsp;</p>

</body>

</html>

Both picture.jpg and index.htm are in the root of the SD card. The card initializes fine, the index.htm is found and is shown in the browser. But where the picture should be just the 'missing image' symbol is shown.
I'm guessing it is a memory problem, but I have no idea how to find out. Or is it something to do with mime-types? I'm lost here...
7  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: April 07, 2013, 07:19:42 am
Extra difficult if the (cable-)modem and router are combined in one box, as they usually are here on most home-connections.
I'm not giving up on UDP hole punching yet, if only because I learn a lot in the process.

Tnx,

Arjen
8  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: April 07, 2013, 06:54:16 am
What do you mean by 'the public side of the router'?
The security is of course a problem and one of the reasons I'd like to communicate with the arduino without port forwarding. Because if I open a port on my router the arduino is also accessible by hackers. I can use a non-standard port and use 'complicated' POST commands so the arduino only responds to a command with some kind of password build in, but if a hacker wants access he/she will get it in the end...
9  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: April 07, 2013, 04:59:31 am
OK, I've done some testing. I've set up my linux box to have Apache also listen to a non standard port. Next I wrote a sketch combining client and server sending and receiving on that same port. So when the client on the Arduino connects to my Linux box, it requests a php script that stores the ip address the Arduino is behind and then a second script attempts to connect on that address. I was hoping that since a connection is made on a specific port the "return call" would be forwarded by my router to the Arduino, but no luck. Apparently this ultra simple TCP-hole punching does not fool my router. It works if I forward the port on my router to the Arduino, so my sketch and php script(s) work, but without the port forwarded the router simply ignores the attempts to connect to the Arduino.
I have looked at the examples from ksduino and bitlash but those are a bit beyond my comprehension. I sort of get the Arduino side of those, but the server side is not clear to me.
Next step is trying to do implement UDP hole punching on the Arduino.
Keep you posted,

A
10  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: April 02, 2013, 10:28:32 am
Sounds interesting, will look in to this as soon as time (read work) allows.

Thanks,

A
11  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: March 31, 2013, 03:29:14 pm
It's not redundant at all, for me this topic is still alive. I just don't have much time at the moment to resolve it.
Thank you for offering a page to use, but I already have a page on a server (my own VPS). The thing is, I don't want the Arduino polling a server all the time. For a real time response to commands send to the Arduino it would need to poll several times per second. This will cause a lot of traffic 24 hrs/day while it only needs to respond to commands maybe 20 times per day. So I need the Arduino to be a server listening  all the time. And to be accessible from outside the LAN. And the ideal situation is getting that without the need for port-forwarding. Hence this topic...

Cheers,

A
12  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: March 22, 2013, 10:01:10 am
Looks like the weekend is swamped smiley-sad But I will post back here if/when I will/will not succeed!

Thanks,

A
13  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: March 22, 2013, 07:30:29 am
I'm afraid time is of the essence here smiley. Not for all functions (i.e. switching between home- or away-temperatures on a thermostat), but one of the things the Arduino controls is a (3-color-led-) light dimmer. You control a slider in the interface and the dimmer is set accordingly. A slight delay is acceptable, but when it takes a second for the light to react it will take several tries before you reach the right mood/color. And having the Arduino poll the server every 100ms seems like overkill to me...
14  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: March 22, 2013, 07:00:03 am
@ billroy: Sounds like this is what I need. I'll look into it as soon as time permits (somehow my job tends to get in the way) and report back here.
@ wildbill: I'll look into that too. Weekend is nearly here so I might have some time to spend on this.


Thanks,

A
15  Topics / Home Automation and Networked Objects / Re: control without port forwarding on: March 22, 2013, 06:35:12 am
That was my first idea of how it should work. But I could not figure out how to send commands from my Linux box to the Arduino when the Arduino is a client. My understanding is that a client only asks from a server. When the server wants to push something to the client (in this case telling the Arduino to change an output port for example) it requires software on the client side that is not doable on an Arduino. So the Arduino also needs to be a server so it can listen to the Linux box.
Or at least that's how I understand it, but I'm not exactly an expert on these matters...
Pages: [1] 2