Go Down

Topic: How can I send data to a web server with GPRS (HTTP or XML) (Read 3824 times) previous topic - next topic

pierrot10

Dear All,

I need to send some data as variable or, at the best, an XML file to a web server.

I have a GPRS Module and the data will be send through the cell phone

The xml file content will be store in a MySQL database.

I am looking for an Arduino Uno or Mini Pro library to send and store data into a MySQL database, BUT I need that library work with different GPRS modules as
this https://www.sparkfun.com/products/9607 or this https://www.sparkfun.com/products/10138



Could you help to make me able to send data to a web server?

Many thank for your help.
Cheers
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

I don't got your point. You have a MySQL DB on an Uno and you are searching for a method to send data from the DB to a webserver?
KaiHeinz Software - http://kaiheinz.de

pierrot10

No, no, sorry,

I have a sketch code will get some value.
The sketch code use the GPRS module to send data to a remote MySQL database / web server.
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

So you want to retrieve the values, create an XML-file and send that to a server where the file(or the values from the file) are stored in a database. Right?
KaiHeinz Software - http://kaiheinz.de

pierrot10

Yes, but with or without XML. The best would be the best.

For now my vision is to send to an php file with will get the value with $_GET and then the PHP script store the value in a databae.
But I am a bit a geniner and I will be interested for the best way to do it.

XML would be very interesting.

Then in resume.
1. Arduino collect the value
2. It use the GPRS module to send the data, it can be an XML file or GET value
3. the web server get and record the value into a database

Do you think there is labrary which do it?
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

pierrot10

#5
Apr 10, 2013, 11:12 pm Last Edit: Apr 10, 2013, 11:16 pm by pierrot10 Reason: 1
Oups, I miss a detail.

it has to be send in a frequently time. For exemple, each 10sec or 20 sec or 60sec...

XML as JSON are interesting for transfering data
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

PeterH

The first link you posted describes the Cellular Shield with SM5100B and includes this link to a document which explains how to use it to send and receive over TCP/IP:

https://www.sparkfun.com/datasheets/CellularShield/SM5100B%20TCPIP%20App%20Note.pdf

As you can see, it's not very complicated but involves you sending AT commands strings to the modem via a serial connection.

You'd need to write or find a set of functions using those capabilities to send and receive data via TCP sockets, and an HTML client to generate the HTTP requests and parse the responses. The HTTP protocol is purely text based and it doesn't take much to perform a simple HTTP transaction, so it'd be reasonable for you to tackle this for yourself.
I only provide help via the forum - please do not contact me for private consultancy.

pierrot10

#7
Apr 11, 2013, 02:09 am Last Edit: Apr 11, 2013, 02:10 am by pierrot10 Reason: 1
Dear Peter,

Thank. But I am trying it since a couple of month and I can not get a result. I have been oft in front of problem regarding the delay between AT request and I am still pending with that issue.

It the reason why I am looking for exemple code wich work in a repeating loop. For exemple send a data each 20 sec.

I fond some library but it still not work....

Would you have an exemple with socksatus?

This is an exemple.I found, but it only send once. After it return an error and it block. The script loop at while(1) loop
Code: [Select]

#include <SoftwareSerial.h>

SoftwareSerial cell(2,3);  // Create a 'fake' serial port. Pin 2 is the Rx pin, pin 3 is the Tx pin. connect this to the GSM module

const String apn = "internet"; // access-point name for GPRS
const String ip = "80.00.000.11"; // IP address of server we're connecting to
const String host = "www.yyy-xxx.net"; // required in HTTP 1.1 - what's the name of the host at this IP address?
const String request = "GET /temp/updatesql.php?lat=1.10&long=1.20&alt=1.30&speed=1.40&course=1.50&date=2013-4-10&time=19:12:00 HTTP/1.1";
const String useragent = "Mozilla/5.0"; // for our purposes the user agent doesn't matter - if I understand correctly it's helpful to use something generic the server will recognize

int GPRS_registered = 0;
int GPRS_AT_ready = 0;

void setup()
{
 Serial.begin(9600);
 
 Serial.println(F(""));
 Serial.println(F("Starting SM5100B Communication..."));
 cell.begin(9600);
 /*
 waitTil("+SIND: 4"); // keep printing cell output til we get "+SIND: 4"
 Serial.println("Module ready");
 */
 
 Serial.println(F("Wait for module is registered (+SIND: 11) and  ready (+SIND: 4) ..."));

 while (GPRS_registered == 0 || GPRS_AT_ready == 0) {
      String ready = getMessage();
     
      if(ready == "+SIND: 1"){
        Serial.println(F("SIM is inserted ..."));
      }
     
      if(ready == "+SIND: 10,\"SM\",1,\"FD\",1,\"LD\",1,\"MC\",1,\"RC\",1,\"ME\",1"){
        Serial.println(F("SIM is ready ..."));
      }
     
      if(ready == "+SIND: 11"){
        GPRS_registered = 1;
        Serial.println(F("Module is registered to network ..."));
      }
     
      if(ready == "+SIND: 3"){
        Serial.println(F("GPRS is partially ready ..."));
      }
     
      if(ready == "+SIND: 4"){
        GPRS_AT_ready = 1;
        Serial.println(F("GPRS is ready ..."));
      }
     
      if(ready == "+SIND: 7"){
        Serial.println(F("Emergency only ..."));
      }
     
   }
   
 
 //waitTil("+SIND: 4"); // keep printing cell output til we get "+SIND: 4"
 Serial.println(F(" "));
 Serial.println(F("READY TO GO"));
 
 
}

void loop()
{
 
 Serial.println(F("Attaching GPRS..."));
 cell.println("AT+CGATT=1");
 waitFor("OK");
 
 Serial.println(F("Setting up PDP Context..."));
 cell.println("AT+CGDCONT=1,\"IP\",\""+apn+"\"");
 waitFor("OK");

 Serial.println(F("Activating PDP Context..."));
 cell.println("AT+CGACT=1,1");
 waitFor("OK");
 
 Serial.println(F("Configuring TCP connection to TCP Server..."));
 cell.println("AT+SDATACONF=1,\"TCP\",\""+ip+"\",80");
 waitFor("OK");
 
 Serial.println(F("Starting TCP Connection..."));
 cell.println("AT+SDATASTART=1,1");
 waitFor("OK");
 
 delay(5000); // wait for the socket to connect
 
 // now we'll loop forever, checking the socket status and only breaking when we connect
 while (1) {
   Serial.println(F("Checking socket status:"));
   cell.println("AT+SDATASTATUS=1"); // we'll get back SOCKSTATUS and then OK
   String sockstat = getMessage();
   waitFor("OK");
   if (sockstat=="+SOCKSTATUS:  1,0,0104,0,0,0") {
     Serial.println(F("Not connected yet. Waiting 1 second and trying again."));
     delay(1000);
   }
   else if (sockstat=="+SOCKSTATUS:  1,1,0102,0,0,0") {
     Serial.println(F("Socket connected"));
     break;
   }
   else {
     Serial.println(F("We didn't expect that."));
     cellOutputForever();
   }
 }
 
 // we're now connected and can send HTTP packets!
 
 int packetLength = 26+host.length()+request.length()+useragent.length(); // 26 is size of the non-variable parts of the packet, see SIZE comments below
 
 Serial.println(F("Sending HTTP packet..."));
 cell.print("AT+SDATATSEND=1,"+String(packetLength)+"\r");
 waitFor('>'); // wait for GSM module to tell us it's ready to recieve the packet
 cell.print(request+"\r\n"); // SIZE: 2
 cell.print("Host: "+host+"\r\n"); // SIZE: 8
 cell.print("User-Agent: "+useragent+"\r\n\r\n"); // SIZE: 16
 cell.write(26); // ctrl+z character: send the packet
 waitFor("OK");
 
 
 // now we're going to keep checking the socket status forever
 // break when the server tells us it acknowledged data
 while (1) {
   cell.println("AT+SDATASTATUS=1"); // we'll get back SOCKSTATUS and then OK
   String s = getMessage(); // we want s to contain the SOCKSTATUS message
   if (s == "+STCPD:1") // this means server sent data. cool, but we want to see SOCKSTATUS, so let's get next message
     s = getMessage();
   //if (s == "+STCPC:1") // this means socket closed. cool, but we want to see SOCKSTATUS, so let's get next message
   //  s = getMessage();
   waitFor("OK");
   
   if (!s.startsWith("+SOCKSTATUS")) {
     Serial.println(F("Wait, this isn't the SOCKSTATUS message!"));
     cellOutputForever(); // something went wrong
   }
   if (checkSocketString(s) == packetLength) // checks that packetLength bytes have been acknowledged by server
     break; // we're good!
   else {
     Serial.println(F("Sent data not yet acknowledged by server, waiting 1 second and checking again."));
     delay(1000);
   }
 }
 Serial.println("Yes! Sent data acknowledged by server!");

 // we could skip the checking of SOCKSTATUS in the above while-loop
 // instead we could just wait for one or both of these:
 //waitTil("+STCPD:1"); // this means data is received
 //waitTil("+STCPC:1"); // this means socket is closed
 
 // TODO: actually check if we received data, don't just do this blindly
// Serial.println(F("Reading data from server..."));
// cell.println("AT+SDATAREAD=1"); // how we read data server has sent
 // WARNING: this might not work - software serial can be too slow for receiving data

//Wait 10 sec and send again the data to server. Of course the data will change within that time
delay(10000);
}


In the next post, I past the following code, the function
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

pierrot10

Here is the following after the loop()
Code: [Select]


/* NOTES
* what is +STIN:1 ?
* to disconnect after transmission: AT+CGACT=0,1 breaks socket. AT+CGATT=0 seems to work more authoritatively?
* AT+SDATASTART=1,0 // close TCP connection
* AT+SDATASTATUS=1 // clear sent/ack bytes from SOCKSTATUS
*/
// keep reading the serial messages we receive from the module
// loop forever until we get a nonzero string ending in \r\n - print and return that.
// TODO: implement a timeout that returns 0?
String getMessage() {
 String s="";
 while(1) {
   if(cell.available()>0) {
     s = s+(char)cell.read();
     if (s.length()>1 && s[s.length()-2]=='\r' && s[s.length()-1]=='\n') { // if last 2 chars are \r\n
       if (s==" \r\n" || s=="\r\n") { // skip these, move on
         s="";
       }
       else { // we have a message!
         Serial.println(s.substring(0,s.length()-2));
         return s.substring(0,s.length()-2);
       }
     }
   }
 }
}

// for eating a single message we expect from the module
// prints out the next message from the module. if it's not the expected value, die
void waitFor(String s) {
 String message=getMessage();
 if (message != s) {
   Serial.println("Wait, that's not what we were expecting. We wanted \""+s+"\"");
   cellOutputForever();
 }
 delay(100); // wait for a tiny bit before sending the next command
}

// keep spitting out messages from the module til we get the one we expect
void waitTil(String s) {
 String message;
 while (1) {
   message = getMessage();
   if (message == s){
     delay(100); // cause we're probably about to send another command
     return;
   }
 }
}

// keep reading characters until we get char c
void waitFor(char c) {
 while(1) {
   if(cell.available()>0) {
     if ((char)cell.read() == c) {
       delay(100);
       return;
     }
   }
 }
}

// if something goes wrong, abort and just display cell module output so we can see error messages
// this will loop forever
void cellOutputForever() {
 Serial.println("Looping forever displaying cell module output...");
 while(1) {
   if(cell.available()>0) {
     Serial.print((char)cell.read());
   }
 }
}

// like above, but in hex, useful for debugging
void cellHexForever() {
 while(1) {
   if(cell.available()>0) {
     char c = (char)cell.read();
//      Serial.print("a char: ");
     Serial.print(c, HEX);
     Serial.print(" ");
     Serial.println(c);
   }
 }
}


// receive string such as "SOCKSTATUS: 1,1,0102,10,10,0"
// 0 is connection id. 1 is whether connected or not. 2 is status (0104 is connecting, 0102 is connected, others)
// 3 is sent bytes. 4 is acknowledged bytes. 5 is "received data counter"
// THIS FUNCTION WILL check that sent bytes == ack bytes, and return that value
// return 0 if they don't match or if amount of data is 0
int checkSocketString(String s) {
 if (socketStringSlice(3,s) == 0)
   return 0;
 else if (socketStringSlice(3,s) == socketStringSlice(4,s))
   return socketStringSlice(3,s);
 else
   return 0;
}

// returns the index of the nth instance of char c in String s
int nthIndexOf(int n, char c, String s) {
 int index=0;
 for (int i=0; i<=n; i++) {
   index = s.indexOf(c,index+1);
 }
 return index;
}

// expects string such as "SOCKSTATUS: 1,1,0102,10,10,0"
// returns nth chunk of data, delimited by commas
int socketStringSlice(int n, String s) {
 String slice = s.substring(nthIndexOf(n-1,',',s)+1,nthIndexOf(n,',',s));
 char cArray[slice.length()+1];
 slice.toCharArray(cArray, sizeof(cArray));
 return atoi(cArray);
}

Is there no an easier way to do it?
Hope you can help. Thank
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

PeterH


After it return an error and it block.


I'm out of practice with GPRS but I'm a bit surprised to see it attaching and creating the PDP context every time through loop. The sketch seems to have plenty of trace statements in it which should tell you what's going on. What's happening in the first and second times through loop()?
I only provide help via the forum - please do not contact me for private consultancy.

pierrot10

It s looping for ever...
The terminal return me message:
Code: [Select]

void waitFor(String s) {
  String message=getMessage();
  if (message != s) {
    Serial.println("Wait, that's not what we were expecting. We wanted \""+s+"\"");
    cellOutputForever();
  }
  delay(100); // wait for a tiny bit before sending the next command
}

But I can not understand why

Some time it's alway showing
Quote

Sent data not yet acknowledged by server, waiting 1 second and checking again

Code: [Select]

while (1) {
    cell.println("AT+SDATASTATUS=1"); // we'll get back SOCKSTATUS and then OK
    String s = getMessage(); // we want s to contain the SOCKSTATUS message
    if (s == "+STCPD:1") // this means server sent data. cool, but we want to see SOCKSTATUS, so let's get next message
      s = getMessage();
    if (s == "+STCPC:1") // this means socket closed. cool, but we want to see SOCKSTATUS, so let's get next message
      s = getMessage();
    waitFor("OK");
   
    if (!s.startsWith("+SOCKSTATUS")) {
      Serial.println(F("Wait, this isn't the SOCKSTATUS message!"));
      cellOutputForever(); // something went wrong
    }
    if (checkSocketString(s) == packetLength) // checks that packetLength bytes have been acknowledged by server
      break; // we're good!
    else {
      Serial.println(F("Sent data not yet acknowledged by server, waiting 1 second and checking again."));
      delay(1000);
    }
  }
Il ne suffit pas de tout savoir, la persévérance, c'est déjà presque tout!

csp10207

hello sir.
i am doing my project on remote data access system.
this project is to transfer the different sensor data value from remote area
several sensors (humidity sensor.tempreture sensor) are conncted to the data logger system.
data logger stores data in every 10 second in a file.
i want to send this file to my email or in a web server.
please help to send file.

Go Up