Verifying that data was sent via SIM800L

I am working on a project that collects data using GPS and RFID. I wish to take this data and then send it Thingspeak via SIM800L

I've broken this down to smaller steps so that I can work out the issues before moving the code to the main program/sketch. For the sketch below, I just want to make sure that data is being sent with 100% reliability. If it works all the time, then I would change the kind of data being sent to be the one I am creating in my main project

My hardware for sending the data

  • Arduino Mega, Rev 3
  • SIM800L GMS module with antenna
  • Power supply for the SIM800L
  • Micro SIM card for the SIM800L

Program objectives

  • Collect some data
  • Send the data to Thingspeak
  • Verify that the data has been sent and received on Thinsgpeak
  • Repeat the above

I checked online for how to use the SIM800L and how to send the data

This link has formed the basis of my project sketch Cellular IoT: Send SIM800/900 GPRS Data to Thingspeak with Arduino

My sketch is as follows:

void setup()
{
  Serial2.begin(9600);               // the GPRS baud rate   
  Serial.begin(9600);    // the GPRS baud rate 
   
  delay(1000);
}
 
void loop()
{
      float h = millis()
      float t = millis()/4; 
      delay(100);   

  if (Serial2.available())
    Serial.write(Serial2.read());
 
  Serial2.println("AT");
  delay(1000);
 
  Serial2.println("AT+CPIN?");
  delay(1000);
 
  Serial2.println("AT+CREG?");
  delay(1000);
 
  Serial2.println("AT+CGATT?");
  delay(1000);
 
  Serial2.println("AT+CIPSHUT");
  delay(1000);
 
  Serial2.println("AT+CIPSTATUS");
  delay(2000);
 
  Serial2.println("AT+CIPMUX=0");
  delay(2000);
 
  ShowSerialData();
 
  Serial2.println("AT+CSTT=\"Safaricom\"");//start task and setting the APN,
  delay(1000);
 
  ShowSerialData();
 
  Serial2.println("AT+CIICR");//bring up wireless connection
  delay(3000);
 
  ShowSerialData();
 
  Serial2.println("AT+CIFSR");//get local IP adress
  delay(2000);
 
  ShowSerialData();
 
  Serial2.println("AT+CIPSPRT=0");
  delay(3000);
 
  ShowSerialData();
  
  Serial2.println("AT+CIPSTART=\"TCP\",\"api.thingspeak.com\",\"80\"");//start up the connection
  delay(6000);
 
  ShowSerialData();
 
  Serial2.println("AT+CIPSEND");//begin send data to remote server
  delay(4000);
  ShowSerialData();
  //API key used below is not the actual one I am using
  String str="GET https://api.thingspeak.com/update?api_key=QNMOITJQRVAHMO&field1=" + String(t) +"&field2="+String(h);
  Serial.println(str); 
  Serial2.println(str);//begin send data to remote server
  
  delay(4000);
  ShowSerialData();
 
  Serial2.println((char)26);//sending
  delay(5000);//waitting for reply, important! the time is base on the condition of internet 
  Serial2.println();
 
  ShowSerialData();
 
  Serial2.println("AT+CIPSHUT");//close the connection
  delay(100);
  ShowSerialData();
} 
void ShowSerialData()
{
  while(Serial2.available()!=0)
  Serial.write(Serial2.read());
  delay(500); 
  
}

Results expected
For the purpose of testing, I am just collecting the milliseconds and sending that.
My expectation is that the data will be sent and received and that this repeats every period of time as seen by the delays - roughly every minute

Actual results

  • Data is sent
  • The thingspeak channel does show the data and graphs it
  • The data being sent increases in value with time
  • However, for every 20 data points that should have been sent, roughly 3 are never received or sent
  • This means that the success rate of sending the data is about 85%

For my main program, I have to send all the data I create, there can be no missing data point.

My thinking is that the SIM module is using 2G technolog and it's not that reliable, hence the data drops. The SIM module has enough power and blinks twice per min when sending data.

Therefore, I feel that I need to introduce some kind of check code to verify that the data that should have been sent was actually received on Thingspeak. If it wasn't sent, then it should try again

Due to the fact that the SIM800L uses serial to communicate, how would I check the incoming serial info and verifiy that the data was received at the website?

When the sketch invokes the ShowSerialData(); function - it shows on the serial monitor the following after data was sent

  • SEND OK
  • A number that increments every time data was sent

I tried to include an if statement that checks if the serial monitor info is SEND OK or not and then do a check with that. But I don't think I got the method right of reading that info

This is what I tried:

void ShowSerialData()
{
  while(Serial2.available()!=0)
  Serial.write(Serial2.read());
  if (Serial2.read()=="SEND OK");
  {
    Serial.prinln("ABC); // just my way of checking that the if statement worked
  }
  delay(500); 
  
}

But the test never goes true

Any guidance on this?

what do you really expect from this?

 Serial.write(Serial2.read());
  if (Serial2.read()=="SEND OK");

Not even mentioning the misplaced semi colon, read() returns the incoming byte when one is available and consumes it: it is removed from the queue. So reading again would read the next byte (or return -1 if none is available), that would never return a full string in any case so you have zero chance for this to work this way.

You need to capture the answer and then parse it.

I would suggest to study Serial Input Basics to handle this

Do you have a free account? ThingSpeak free accounts have a maximum update frequency of once per 15 seconds.
I saw a delay(4000);
But that wont be long enough. cAn you try adding more delay between writes?

J-M-L:
what do you really expect from this?

 Serial.write(Serial2.read());

if (Serial2.read()=="SEND OK");



Not even mentioning the misplaced semi colon, read() returns the incoming byte when one is available and consumes it: it is removed from the queue. So reading again would read the next byte (or return -1 if none is available), that would never return a full string in any case so you have zero chance for this to work this way.

You need to capture the answer and then parse it.

I would suggest to study [Serial Input Basics](http://forum.arduino.cc/index.php?topic=396450.0) to handle this

Thank you for guiding me to that link - it helped a lot in moving me forward
I was able to read the data coming from the SIM800L module and do a test to see if the words "SEND OK" were received or not

cstapels:
Do you have a free account? ThingSpeak free accounts have a maximum update frequency of once per 15 seconds.
I saw a delay(4000);
But that wont be long enough. cAn you try adding more delay between writes?

Yes, I am using a free account. I am aware of the update frequency limitation but the whole loop takes longer than 15 seconds, so technically speaking, I am updating values at a duration longer than 15 seconds

Chintan_Gohel:
I was able to read the data coming from the SIM800L module and do a test to see if the words "SEND OK" were received or not

great.

I've been working on this further and realised that the way I wanted to test if the data was sent or not will not work

The issue here is that sometimes the data is sent and it is received on Thingspeak but the confirmation message isn't received.

Or sometimes the confirmation message is received after the loop is over and the next loop has started

Or sometimes the confirmation message is only half received "SEND " instead of "SEND OK"

Today I did a test with 11 sets of data to be sent and only 6 were actually sent, meaning 5 were lost

If I do put in a test to check if the SEND OK message wa receiveed, then it will give wrong results and I might end up sending the data twice or thrice because the SEND OK message wasn't received correctly or on time

Any ideas here?

The fact that you don’t receive confirmation needs to be investigated, it might be a bug in the way you handle the asynchronous communication.

If it’s not a major issue to confirm or not a transaction you might envision forgetting all about having an acknowledgment. If it’s important You need to decide what to do when an acknowledgement is lost - you might need to strengthen the protocol to have a server side to rollback a transaction.

You also need to decide if the sending of the information is totally independent from the ack and you handle both as asynchronous tasks using queues or if you make it synchronous and one message at a time: Send / wait ack or timeout / handle status (retry sending?) and loop

a state machine could help structure managing this.

J-M-L:
The fact that you don’t receive confirmation needs to be investigated, it might be a bug in the way you handle the asynchronous communication.

To be honest, I'm not that comfortable with the coding since I don't understand parts of it and the code has been copied from another site. I've tested 2-3 codes for sending info to thingspeak using a SIM module and this works. It's just that at times the data is not sent and I'm not sure if it because the technology uses 2G or because of the way the code has been put. Plus the code uses delays and I know there must be a better way for the delays to be put, but I'm not sure how to do it yet

If it's not a major issue to confirm or not a transaction you might envision forgetting all about having an acknowledgment. If it's important You need to decide what to do when an acknowledgement is lost - you might need to strengthen the protocol to have a server side to rollback a transaction.

It is definitely important that all the data be sent. The data includes transaction info and if the website does not receive it, then the service provider doens't paid

So what to do when the data is sent, but the ackowledgement is lost? If I had done a checking system to check if the data was sent or not and the confirmation message was missing, then the data would be sent again. But this would mean the website receives the same info twice. So yes, on the server side there would have to be a test to check if the info has been repeated or not. The way I envisage this is to give each transaction a unique number and on the server side, check for the number and block repeated entries