Server and client at the same time

Using Arduino Networking, Protocols, and Devices
#1

I have a:
Arduino Mega 2560
Ethernet Shield rev 3
Sensor

What i want to do is to host a webpage on the arduino , and periodicly make post request to ThingSpeak.

I'm confused how should i go with this. Should i make 2 ethernet clients?
Can somebody please post a example with the default arduino web server example and this POST request:

void updateThingSpeak(String tsData)
{
  if (client.connect(thingSpeakAddress, 80))
  {         
    client.print("POST /update HTTP/1.1\n");
    client.print("Host: api.thingspeak.com\n");
    client.print("Connection: close\n");
    client.print("X-THINGSPEAKAPIKEY: "+writeAPIKey+"\n");
    client.print("Content-Type: application/x-www-form-urlencoded\n");
    client.print("Content-Length: ");
    client.print(tsData.length());
    client.print("\n\n");

    client.print(tsData);
#2

I'm confused how should i go with this. Should i make 2 ethernet clients?

You create a client to make a request. The server "creates" a client when a client connects to the server.

zoomkat periodically posts code for the Arduino as client and server. I'm sure that you can find it, if you try.

#3

How secure is the network going to be? zoomkat’s server code has some problems if connected to the internet (can lock up), and his client code has a bit of trouble on a busy network or slow server (gets a lot of “funny y” characters).

If not secure, you can combine my web server and web client code examples relatively easily. The server has enough error checking and fault tolerance to not lock up or crash if abused by malevolent clients, and the client will not lock up if the connection breaks or the server stalls. You can only combine both of mine if using a Mega or better. Not enough memory on an Uno.
http://playground.arduino.cc/Code/WebClient
http://playground.arduino.cc/Code/WebServerST

edit: Don’t use a delay() call to time the client call to the ThingSpeak server. Use a millis() call using a “blink without delay” type algorithm.

I attached my best server code combined with my best client code. It sends a really basic GET request to the thingspeak api every 30 seconds. You can change that part to a POST request if you want.

WebServerClient.ino (14 KB)

#4

What i want to do is to host a webpage on the arduino , and periodicly make post request to ThingSpeak.

Below is the basic client/server code you might try to see if it works in your setup.

//zoomkat 10-02-14, combined client and server
//simple button GET with iframe code
//for use with IDE 1.0
//open serial monitor and send a g to test client GET and
//see what the arduino client/server receives
//web page buttons make pins high/low
//use the ' in html instead of " to prevent having to escape the "
//address will look like http://192.168.1.102:84 when submited
//for use with W5100 based ethernet shields

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //assign arduino mac address
byte ip[] = {192, 168, 1, 102 }; // ip in lan assigned to arduino
byte gateway[] = {192, 168, 1, 1 }; // internet access via router
byte subnet[] = {255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port arduino server will use
EthernetClient client;
char serverName[] = "checkip.dyndns.com"; // (DNS) dyndns web page server
//byte serverName[] = { 208, 104, 2, 86 }; // (IP) zoomkat web page server IP address

String readString; //used by server to capture GET request 

//////////////////////

void setup(){

  pinMode(5, OUTPUT); //pin selected to control
  pinMode(6, OUTPUT); //pin selected to control
  pinMode(7, OUTPUT); //pin selected to control
  pinMode(8, OUTPUT); //pin selected to control

  //pinMode(5, OUTPUT); //pin 5 selected to control
  Ethernet.begin(mac,ip,gateway,gateway,subnet); 
  server.begin();
  Serial.begin(9600); 
  Serial.println(F("server/client 1.0 test 9/02/14")); // keep track of what is loaded
  Serial.println(F("Send a g in serial monitor to test client")); // what to do to test client
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) 
  {
    byte inChar;
    inChar = Serial.read();
    if(inChar == 'g')
    {
      sendGET(); // call client sendGET function
    }
  }  

  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {

          //store characters to string 
          readString += c; 
          //Serial.print(c);
        } 

        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.print(readString); //print to serial monitor for debuging 

            //now output HTML data header
          if(readString.indexOf('?') >=0) { //don't send new page
            client.println(F("HTTP/1.1 204 Zoomkat"));
            client.println();
            client.println();  
          }
          else {   
            client.println(F("HTTP/1.1 200 OK")); //send new page on browser request
            client.println(F("Content-Type: text/html"));
            client.println();

            client.println(F("<HTML>"));
            client.println(F("<HEAD>"));
            client.println(F("<TITLE>Arduino GET test page</TITLE>"));
            client.println(F("</HEAD>"));
            client.println(F("<BODY>"));

            client.println(F("<H1>Zoomkat's simple Arduino 1.0 button</H1>"));

            // DIY buttons
            client.println(F("Pin5"));
            client.println(F("<a href=/?on2 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off3 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin6"));
            client.println(F("<a href=/?on4 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off5 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin7"));
            client.println(F("<a href=/?on6 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off7 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin8"));
            client.println(F("<a href=/?on8 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off9 target=inlineframe>OFF</a>

")); 

            client.println(F("Pins"));
            client.println(F("&nbsp;<a href=/?off2468 target=inlineframe>ALL ON</a>")); 
            client.println(F("&nbsp;<a href=/?off3579 target=inlineframe>ALL OFF</a>")); 

            client.println(F("<IFRAME name=inlineframe style='display:none'>"));          
            client.println(F("</IFRAME>"));

            client.println(F("</BODY>"));
            client.println(F("</HTML>"));
          }

          delay(1);
          //stopping client
          client.stop();

          ///////////////////// control arduino pin
          if(readString.indexOf('2') >0)//checks for 2
          {
            digitalWrite(5, HIGH);    // set pin 5 high
            Serial.println(F("Led 5 On"));
            Serial.println();
          }
          if(readString.indexOf('3') >0)//checks for 3
          {
            digitalWrite(5, LOW);    // set pin 5 low
            Serial.println(F("Led 5 Off"));
            Serial.println();
          }
          if(readString.indexOf('4') >0)//checks for 4
          {
            digitalWrite(6, HIGH);    // set pin 6 high
            Serial.println(F("Led 6 On"));
            Serial.println();
          }
          if(readString.indexOf('5') >0)//checks for 5
          {
            digitalWrite(6, LOW);    // set pin 6 low
            Serial.println(F("Led 6 Off"));
            Serial.println();
          }
          if(readString.indexOf('6') >0)//checks for 6
          {
            digitalWrite(7, HIGH);    // set pin 7 high
            Serial.println(F("Led 7 On"));
            Serial.println();
          }
          if(readString.indexOf('7') >0)//checks for 7
          {
            digitalWrite(7, LOW);    // set pin 7 low
            Serial.println(F("Led 7 Off"));
            Serial.println();
          }     
          if(readString.indexOf('8') >0)//checks for 8
          {
            digitalWrite(8, HIGH);    // set pin 8 high
            Serial.println(F("Led 8 On"));
            Serial.println();
          }
          if(readString.indexOf('9') >0)//checks for 9
          {
            digitalWrite(8, LOW);    // set pin 8 low
            Serial.println(F("Led 8 Off"));
            Serial.println();
          }         

          //clearing string for next read
          readString="";

        }
      }
    }
  }
} 

//////////////////////////
void sendGET() //client function to send and receive GET data from external server.
{
  if (client.connect(serverName, 80)) {
    Serial.println(F("connected"));
    client.println(F("GET / HTTP/1.1"));
    client.println(F("Host: checkip.dyndns.com"));
    client.println(F("Connection: close"));
    client.println();
  } 
  else {
    Serial.println(F("connection failed"));
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    readString += c; //places captured byte in readString
  }

  //Serial.println();
  client.stop(); //stop client
  Serial.println(F("client disconnected."));
  Serial.println(F("Data from server captured in readString:"));
  Serial.println();
  Serial.print(readString); //prints readString to serial monitor 
  Serial.println();
  Serial.println(F("End of readString"));
  Serial.println(F("=================="));
  Serial.println();
  readString=""; //clear readString variable

}
#5

zoomkat:
Below is the basic client/server code you might try to see if it works in your setup.

Thanks for the swift response.
However , when i try to replace your sendGet function with a function that sends data to ThingSpeak , the program refuses to connect to ThingSpeak.

I’m basically trying to stitch up two codes together , and failing miserably.
Here is the ThingSpeak example CLICK

and here is the code i’m trying

//zoomkat 12-08-11, combined client and server
//simple button GET with iframe code
//for use with IDE 1.0
//open serial monitor and send an g to test client and
//see what the arduino client/server receives
//web page buttons make pin 4 high/low
//use the \ slash to escape the " in the html
//address will look like http://192.168.1.102:84 when submited
//for use with W5100 based ethernet shields

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
IPAddress ip(192, 168, 0, 197); // ip in lan
IPAddress gateway(192, 168, 0, 1); // internet access via router
IPAddress subnet(255, 255, 255, 0); //subnet mask

// ThingSpeak Settings
char thingSpeakAddress[] = "api.thingspeak.com";
String writeAPIKey = "2M6SFMKICXR7MU1A";
const int updateThingSpeakInterval = 4 * 1000;

// Variable Setup
long lastConnectionTime = 0;
boolean lastConnected = false;
int failedCounter = 0;

EthernetServer server(84); //server port
EthernetClient client;
String readString;

//////////////////////

void setup() {

  pinMode(4, OUTPUT); //pin selected to control
  Ethernet.begin(mac, ip, subnet, gateway);
  server.begin();
  Serial.begin(9600);
  Serial.println("server/client 1.0 test 12/08/11"); // keep track of what is loaded
  Serial.println("Send an g in serial monitor to test client"); // what to do to test client
}

void loop() {
  
  // Print Update Response to Serial Monitor
  if (client.available())
  {
    char c = client.read();
    Serial.print(c);
  }

  // Disconnect from ThingSpeak
  if (!client.connected() && lastConnected)
  {
    Serial.println("...disconnected");
    Serial.println();
    
    client.stop();
  }
  
  
  if(!client.connected() && (millis() - lastConnectionTime > updateThingSpeakInterval))
  {
    Serial.println("sendPost function started");
    sendPost();
  }

  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {

          //store characters to string
          readString += c;
          //Serial.print(c);
        }

        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.println(readString); //print to serial monitor for debuging

          //now output HTML data header
          if (readString.indexOf('?') >= 0) { //don't send new page
            client.println("HTTP/1.1 204 ");
            client.println();
            client.println();
          }
          else {
            client.println("HTTP/1.1 200 OK"); //send new page
            client.println("Content-Type: text/html");
            client.println();

            client.println("<HTML>");
            client.println("<HEAD>");
            client.println("<TITLE>Arduino test page</TITLE>");
            client.println("</HEAD>");
            client.println("<BODY>");

            client.println("<H1>THIS IS A TEST</H1>");

            client.println("</BODY>");
            client.println("</HTML>");
          }

          delay(1);
          //stopping client
          client.stop();

          
          readString = "";

        }
      }
    }
  }
}


void sendPost() {
  Serial.println("function initiated");
  client.stop();

  String tsData = String(analogRead(A0), DEC);

  if (client.connect(thingSpeakAddress, 80))
  {
    client.print("POST /update HTTP/1.1\n");
    client.print("Host: api.thingspeak.com\n");
    client.print("Connection: close\n");
    client.print("X-THINGSPEAKAPIKEY: " + writeAPIKey + "\n");
    client.print("Content-Type: application/x-www-form-urlencoded\n");
    client.print("Content-Length: ");
    client.print(tsData.length());
    client.print("\n\n");
    client.print("field1="); //// is this the prob?
    client.print(tsData);
    
    
    lastConnectionTime = millis();
    
    if (client.connected())
    {
      Serial.println("Connecting to ThingSpeak...");
      Serial.println();
      
      failedCounter = 0;
    }
    else
    {
      failedCounter++;
  
      Serial.println("Connection to ThingSpeak failed ("+String(failedCounter, DEC)+")");   
      Serial.println();
    }
    
  }
  else
  {
    failedCounter++;
    
    Serial.println("Connection to ThingSpeak Failed ("+String(failedCounter, DEC)+")");   
    Serial.println();
    
    lastConnectionTime = millis(); 
  }



 


}
#6

Despite my warnings about zoomkat's code, you are surprised it has bugs?

Here is one in the server section. If you do not specify port 84 on the web browser address, it won't connect.

// change this
EthernetServer server(84); //server port
// to this
EthernetServer server(80); //server port

Here is another in the client section. If the server stalls or the connection breaks, this becomes an endless loop and "locks up" here. If the server is slow or busy, you may get a bunch of "funny y" characters. And if the server response is large, which most are, it will run out of SRAM storing characters in the readString variable.

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    readString += c; //places captured byte in readString
  }

Here is one in your code.

// change this
  Ethernet.begin(mac, ip, subnet, gateway);
// to this
  Ethernet.begin(mac, ip, gateway, gateway, subnet);

Here is another.

// replace this
  if (client.connect(thingSpeakAddress, 80))
// with this.
  if (client.connect(thingSpeakAddress, 80) == 1)

edit: Even though in the past I have supported the use of the F() function with the client.print() functions to conserve SRAM, it will choke some routers, causing a connection abort. And it is slow! It sends one character per packet. :frowning:

I no longer use or recommend those in combination.

The F() function is ok when used with the Serial.print() functions because the Serial functions do not use packets.

#7

SurferTim:
Despite my warnings about zoomkat's code, you are surprised it has bugs?

I appreciate your response.
However you don't seem to understand my total lack of knowledge in this area.
I do not care how "stable" the server is , or how "protected it is" I'll be using it in my own local network most of the time.

I'm aware that i have to specify the port if it is not 80.
The while loops are not in the code i posted , these are in zoomkat's code.
changing the ethernet.begin parameters seemed to brake the program totally.
I've put a "==1"

The code you've posted several posts back is too complicated for me , and i don't understand it.This is mostly because my lack of knowledge and experience in this area.

If you have the time , could you post the simplest code that will host a webpage and upload data from A0 to ThingSpeak. Comments will be greatly appreciated too.

here is the example that ThingSpeak give to be able to upload data to their servers GitHub

I tried to do this several times , but no matter how i stitch up the example web page code and the code provided by ThingSpeak , either the webpage or the uploading doesn't seem to work.

I understand that this will be time consuming for you , but i'm out of options at this point.

Cheers.

#8

However , when i try to replace your sendGet function with a function that sends data to ThingSpeak , the program refuses to connect to ThingSpeak.

I looked at the github thingspeak code and plugged some of it into my test code below. Did a quick test and the webserver part of the code works, and the client connects to the thingspeak server. I don’t have a thingspeak account, so the dummy data I used returns 400 Bad Request. You might try putting in some simple valid thingspeak data to see if thingspeak updates. If that works, other things might be added.

//zoomkat 10-02-14, combined client and server
//simple button GET with iframe code
//for use with IDE 1.0
//open serial monitor and send a g to test client GET and
//see what the arduino client/server receives
//web page buttons make pins high/low
//use the ' in html instead of " to prevent having to escape the "
//address will look like http://192.168.1.102:84 when submited
//for use with W5100 based ethernet shields

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //assign arduino mac address
byte ip[] = {192, 168, 1, 102 }; // ip in lan assigned to arduino
byte gateway[] = {192, 168, 1, 1 }; // internet access via router
byte subnet[] = {255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port arduino server will use
//char serverName[] = "checkip.dyndns.com"; // test web page server

char thingSpeakAddress[] = "api.thingspeak.com";
EthernetClient client;
String writeAPIKey = "XXXMX2WYYR0EVZZZ";
String tsData = "hello";  //dummy data
String readString; //used by server to capture POST request 

//////////////////////

void setup(){

  pinMode(5, OUTPUT); //pin selected to control
  pinMode(6, OUTPUT); //pin selected to control
  pinMode(7, OUTPUT); //pin selected to control
  pinMode(8, OUTPUT); //pin selected to control

  //pinMode(5, OUTPUT); //pin 5 selected to control
  Ethernet.begin(mac,ip,gateway,gateway,subnet); 
  server.begin();
  Serial.begin(9600); 
  Serial.println(F("server/client 1.0 test 9/02/14")); // keep track of what is loaded
  Serial.println(F("Send a g in serial monitor to test client")); // what to do to test client
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) 
  {
    byte inChar;
    inChar = Serial.read();
    if(inChar == 'g')
    {
      sendGET(); // call client sendGET function
    }
  }  

  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {

          //store characters to string 
          readString += c; 
          //Serial.print(c);
        } 

        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.print(readString); //print to serial monitor for debuging 

            //now output HTML data header
          if(readString.indexOf('?') >=0) { //don't send new page
            client.println("HTTP/1.1 204 Zoomkat");
            client.println();
            client.println();  
          }
          else {   
            client.println(F("HTTP/1.1 200 OK")); //send new page on browser request
            client.println(F("Content-Type: text/html"));
            client.println();

            client.println(F("<HTML>"));
            client.println(F("<HEAD>"));
            client.println(F("<TITLE>Arduino GET test page</TITLE>"));
            client.println(F("</HEAD>"));
            client.println(F("<BODY>"));

            client.println(F("<H1>Zoomkat's simple Arduino 1.0 button</H1>"));

            // DIY buttons
            client.println(F("Pin5"));
            client.println(F("<a href=/?on2 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off3 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin6"));
            client.println(F("<a href=/?on4 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off5 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin7"));
            client.println(F("<a href=/?on6 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off7 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin8"));
            client.println(F("<a href=/?on8 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off9 target=inlineframe>OFF</a>

")); 

            client.println(F("Pins"));
            client.println(F("&nbsp;<a href=/?off2468 target=inlineframe>ALL ON</a>")); 
            client.println(F("&nbsp;<a href=/?off3579 target=inlineframe>ALL OFF</a>")); 

            client.println(F("<IFRAME name=inlineframe style='display:none'>"));          
            client.println(F("</IFRAME>"));

            client.println(F("</BODY>"));
            client.println(F("</HTML>"));
          }

          delay(1);
          //stopping client
          client.stop();

          ///////////////////// control arduino pin
          if(readString.indexOf('2') >0)//checks for 2
          {
            digitalWrite(5, HIGH);    // set pin 5 high
            Serial.println(F("Led 5 On"));
            Serial.println();
          }
          if(readString.indexOf('3') >0)//checks for 3
          {
            digitalWrite(5, LOW);    // set pin 5 low
            Serial.println(F("Led 5 Off"));
            Serial.println();
          }
          if(readString.indexOf('4') >0)//checks for 4
          {
            digitalWrite(6, HIGH);    // set pin 6 high
            Serial.println(F("Led 6 On"));
            Serial.println();
          }
          if(readString.indexOf('5') >0)//checks for 5
          {
            digitalWrite(6, LOW);    // set pin 6 low
            Serial.println(F("Led 6 Off"));
            Serial.println();
          }
          if(readString.indexOf('6') >0)//checks for 6
          {
            digitalWrite(7, HIGH);    // set pin 7 high
            Serial.println(F("Led 7 On"));
            Serial.println();
          }
          if(readString.indexOf('7') >0)//checks for 7
          {
            digitalWrite(7, LOW);    // set pin 7 low
            Serial.println(F("Led 7 Off"));
            Serial.println();
          }     
          if(readString.indexOf('8') >0)//checks for 8
          {
            digitalWrite(8, HIGH);    // set pin 8 high
            Serial.println(F("Led 8 On"));
            Serial.println();
          }
          if(readString.indexOf('9') >0)//checks for 9
          {
            digitalWrite(8, LOW);    // set pin 8 low
            Serial.println(F("Led 8 Off"));
            Serial.println();
          }         

          //clearing string for next read
          readString="";

        }
      }
    }
  }
} 

//////////////////////////
void sendGET() //client function to send and receive GET data from external server.
{
  if (client.connect(thingSpeakAddress, 80)) { 
  client.print("POST /update HTTP/1.1\n");
    client.print("Host: api.thingspeak.com\n");
    client.print("Connection: close\n");
    client.print("X-THINGSPEAKAPIKEY: "+writeAPIKey+"\n");
    client.print("Content-Type: application/x-www-form-urlencoded\n");
    client.print("Content-Length: ");
    client.print(tsData.length());
    client.print("\n\n");

    client.print(tsData);
  } 
  else {
    Serial.println(F("connection failed"));
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    readString += c; //places captured byte in readString
  }

  //Serial.println();
  client.stop(); //stop client
  Serial.println(F("client disconnected."));
  Serial.println(F("Data from server captured in readString:"));
  Serial.println();
  Serial.print(readString); //prints readString to serial monitor 
  Serial.println();
  Serial.println(F("End of readString"));
  Serial.println(F("=================="));
  Serial.println();
  readString=""; //clear readString variable

}
#9

zoomkat:
I looked at the github thingspeak code and plugged some of it into my test code below. Did a quick test and the webserver part of the code works, and the client connects to the thingspeak server. I don't have a thingspeak account, so the dummy data I used returns 400 Bad Request. You might try putting in some simple valid thingspeak data to see if thingspeak updates. If that works, other things might be added.

The code works GREAT , thank you for the time and dedication you've put into my problem. I was stuck at this for so long.
I am eternally grateful!

#10

The gethub code has a client three attempt reset feature that might be worth adding to your code.

#11

In the attached code (it now exceeds 9000 characters) I added the network reconnection code from the github thingspeak example code. It seems to work ok. I can disconnect the network cable from the arduino until the thingspeak connections fail, then reconnect the cable and the arduino will start reconnecting to thingspeak. The web server restarts too.

Client_Server_thingspeak.ino (9.5 KB)

#12