ESP8266-01 Wifi Shield ceases to be available after around ~45 cycles of data transmission-- why?

Background: I am attempting to collect air pollution data hourly for 3 months at a remote site. I am using an Arduino UNO for the board, a PMS5003 for the sensor, and an ESP8266-01 Wifi Shield both to sync time with an NTP server (https://pool.ntp.org/) and to upload collected data to Thingspeak. I am using an ESP for timesync rather than an RTC module because I would like to dedicate the 5V power solely to the sensor for better accuracy.

Circuit can be found in this video at 1:01: Getting your Arduino online - Using the ESP-01 and Blynk - YouTube

Link to full code (quite long and complex, and the area where the problem occurs is included further below): air-sensing-hourly/PMSLibraryFullCode.ino at main · throwaway3141/air-sensing-hourly · GitHub

My current method: Every time the clock hits 55 minutes, the sensor should "wake up", giving it 5 minutes of buffer time before it reads data. At the 60 minute mark (aka the start of the new hour) the sensor should then read data, send it, and go back to sleep. As such, I sync the Arduino to realtime from the NTP server every 1 minute in the loop*,* a frequency which is convenient for hourly data recording and slow enough for the ESP to relay without lag. Once the current minute is 54 or 59, I then switch to a short manual delay using millis() in order to align the remaining seconds exactly with 55 or 60 minutes, and then perform an action (wake or read-sleep). I am aware that this may not be the best way to code this output-- please tell me if there are easier/better solutions.

The problem: After around 45 minutes (so around 45 timesync command cycles), the ESP8266-01 simply fails to become available. In other words, after running the timesync commands enough times, the ESP seems to shut down, despite working in the first few iterations. Specifically, this occurs in the function which retrieves time from the NTP server: the line while(!esp8266.available()) {} never leaves the loop.

unsigned long getTimeEpochUnix() {
  unsigned long secsSince1900 = 0UL; //Since we only care about time of day and not date, this isn't really necessary. But we might as well get the full time right anyway

  sendCommand("AT+CIPSTART=\"UDP\",\""+ TIME_HOST +"\","+ TIME_PORT, 15, "OK"); //NOT STRICTLY NECESSARY IF ALREADY CONNECTED?
  sendCommand("AT+CIPSEND=48", 5, "OK");
  
  emptyESP_RX(1000UL);
  esp8266.write((char*)&ntpFirstFourBytes, NTP_PACKET_SIZE);  // Request

  // skip  AT command answer ("Recv 48 bytes\r\n\r\nSEND OK\r\n\r\n+IPD,48:")
  waitForString(":", 1000UL);

  // read the NTP packet, extract the TRANSMIT TIMESTAMP in Seconds from bytes 40,41,42,43
  for (int i = 0; i < NTP_PACKET_SIZE; i++) {
    unsigned long cT = 0;
    unsigned long lT = millis();
    while (!esp8266.available()) {}
    int c = esp8266.read();
    if ((i >= 40) && (i < 44)) secsSince1900 = (secsSince1900 << 8) + (unsigned long)((uint8_t)(c & (int)0x00FF));  // Read the integer part of sending time
    else if (i == 44) secsSince1900 += (((uint8_t)c) > SECONDROUNDINGTHRESHOLD ? 1 : 0);
  }

  epochUnix = secsSince1900 - SEVENTYYEARS; // subtract seventy years. Again, since we only care time of day, this isn't strictly necessary.


  sendCommand("AT+CIPCLOSE",5,"OK");
  return epochUnix + UTC_DELTA;
}

I initially thought the problem to be with hardware voltage, but after checking resistors and confirming power output with a voltmeter, I doubt this is the case. I have also tried executing AT+RESET in the while loop() after a certain period of unavailability, and although this occasionally works, it is too inconsistent to be a feasible solution. It could be possible that the error lies somewhere in the methodology of the specific code used to unpack the timeserver data: this code was largely copied off a demo, and I'm somewhat unsure how it works.

Thank you for reading; any help would be massively appreciated.

there is an AT command do get the NTP time

Here is the "schematic" :

and the code:

#include "PMS.h"
#include "SoftwareSerial.h"



// BASIC VARIABLE SETUP ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


// Hardware --------------------------------
SoftwareSerial SerialPMS(2, 3);
PMS pms(SerialPMS);
SoftwareSerial esp8266(10,11); 

#define hardRestPIN 22 // J-M-L thing for reset, might as well include

// General ESP
String AP = //Wifi username
String PASS = // Wifi password

// Thingspeak --------------------------------
String THINGSPEAK_API = "41PA69WJUNTNSWY1"; // Write API KEY
String THINGSPEAK_HOST = "api.thingspeak.com";
String THINGSPEAK_PORT = "80";

// Time --------------------------------
#define SECONDROUNDINGTHRESHOLD 115 //Adjust depending on transmission delay
// after a successful querry we will get a 48 byte answer from the NTP server.
// To understand the structure of an NTP querry and answer
// see http://www.cisco.com/c/en/us/about/press/internet-protocol-journal/back-issues/table-contents-58/154-ntp.html
// so if we want to read the "Transmit Timestamp" then we need to
// Read the integer part which are bytes 40,41,42,43
// if we want to round to the nearest second if we want some accuracy
// Then fractionary part are byte 44,45,46,47
// if it is greater than 500ms byte 44 will be > 128
// and thus by only checking byte 44 of the answer we can round to the next second;
// 90% of the NTP servers have network delays below 100ms
// We can also account for an assumed averaged network delay of 50ms, and thus instead of
// comparing with 128 you can compare with (0.5s - 0.05s) * 256 = 115;
#define SEVENTYYEARS 2208988800UL
#define NUMBEROFSECONDSPERDAY 86400UL
#define NUMBEROFSECONDSPERHOUR 3600UL
#define NUMBEROFSECONDSPERMINUTE 60UL
const long UTC_DELTA = -14400; // (-4 hours offset for EDT) * (3600 seconds per hour)

const unsigned long ntpFirstFourBytes = 0xEC0600E3;  // NTP request header, first 32 bits
const uint8_t NTP_PACKET_SIZE = 48; // NTP time stamp is in the first 48 bytes of the message
String TIME_HOST = "pool.ntp.org";
String TIME_PORT = "123"; // port for communication

uint8_t currentHour, currentMinute, currentSecond;
unsigned long epochUnix;


// Code main loop execution --------------------------------
unsigned long currentTime;
unsigned long currentDelay;
static unsigned long lastTime = 0;
static bool justWoke = false;
static bool justRead = true;
static bool hasSetDelay = false;



// setup() and other setup functions  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
void setup()
{

  //Serial for printing, esp8266 for time and thingspeak, SerialPMS for sensor  
  Serial.begin(9600);
  esp8266.begin(9600);
  SerialPMS.begin(9600); 

  setupESP();
  
  setupSensor();

  updateCurrentTime();

  Serial.print("Initial ");
  displayTime();

}


void setupESP() {
  esp8266.listen(); //Switch listening over to esp serial
  
  //Resetting of the device
  emptyESP_RX(1000UL);
  digitalWrite(hardRestPIN, LOW);
  delay(100);
  digitalWrite(hardRestPIN, HIGH);

  sendCommand("AT",5,"OK"); // Checks that ESP is working
  sendCommand("AT+RST",10,"invalid"); //"magic fix" (see google doc)
  sendCommand("AT+CWMODE=1",5,"OK"); // Sets wifi mode
  //Connects to an AP

  sendCommand("AT+CWQAP",5,"OK");
  sendCommand("AT+CWJAP=\""+ AP +"\",\""+ PASS +"\"",20,"OK");  //Connects to an AP
  sendCommand("AT+CIPMUX=0",5,"OK"); 
}

void setupSensor() {
  SerialPMS.listen(); //Switch listening over to sensor serial
  pms.passiveMode();
  // Default state after sensor power, but undefined after ESP restart e.g. by OTA flash, so we have to manually wake up the sensor for sure.
  // Some logs from bootloader is sent via Serial port to the sensor after power up. This can cause invalid first read or wake up so be patient and wait for next read cycle.
  pms.wakeUp();
}

// loop() ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
void loop()
{ 
  

  if (millis() - lastTime >= 60000UL) { //updateCurrentTime every minute. Starts off by going a minute from time at code execution. 
    lastTime = millis();

    updateCurrentTime();
    displayTime();

  }

  if (currentMinute == 54 && !justWoke) { //wake
    //Serial.println("Ok");
    if (!hasSetDelay) {
      Serial.println("Awaiting wake: " + String(currentDelay));
      lastTime = millis();
      currentDelay = (60 - currentSecond) * 1000;
      hasSetDelay = true;
    }

    if ((millis() - lastTime >= currentDelay) || currentDelay == 0) { //==0 is a catchall
      SerialPMS.listen(); //Switch listening over to sensor serial
      
      Serial.print("Waking up. "); 
      pms.wakeUp();

      updateCurrentTime();
      displayTime();
      
      justRead = false;
      justWoke = true;

      hasSetDelay = false;
      lastTime = millis(); //optional?
    }


  } else if (currentMinute == 59 && !justRead) {
    
    if (!hasSetDelay) {
      Serial.println("Awaiting read:");
      lastTime = millis();
      currentDelay = (60 - currentSecond) * 1000;
      hasSetDelay = true;
    }

    if ((millis() - lastTime >= currentDelay) || currentDelay == 0) {
      SerialPMS.listen(); //Switch listening over to sensor serial

      Serial.println("Send read request...");
      readData();
      Serial.println("Data printed and going to sleep.");
      pms.sleep();

      updateCurrentTime();
      displayTime();
    
      justWoke = false;
      justRead = true;

      hasSetDelay = false;
      lastTime = millis(); //optional?

    }

  }

//10 seconds between waking up and reading data
//20 seconds between sleep and waking up
}

// readData() ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
void readData()
{
  PMS::DATA data;


  // Clear buffer (removes potentially old data) before read. Some data could have been also sent before switching to passive mode.
  while (SerialPMS.available()) { SerialPMS.read(); } //Changed this to SerialPMS from Serial. Should fix things.

  pms.requestRead();

  Serial.print("Reading data...");
  displayTime();
  if (pms.readUntil(data))
  {
    Serial.println();
    
    // Get data first
    int data10 = data.PM_AE_UG_1_0;
    int data25 = data.PM_AE_UG_2_5;
    int data100 = data.PM_AE_UG_10_0;

    Serial.print("PM 1.0 (ug/m3): ");
    Serial.println(String(data10));
    Serial.print("PM 2.5 (ug/m3): ");
    Serial.println(String(data25));
    Serial.print("PM 10.0 (ug/m3): ");
    Serial.println(String(data100));

    sendThingspeakData(data10, data25, data100);
  }
  else
  {
    Serial.println("No data.");
  }

}


// Thingspeak function ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

void sendThingspeakData(int data10, int data25, int data100) {
  esp8266.listen(); // Switch listening over to esp serial

  //Sending a specific PMS value to a specific field
  // set https request copy pasted from thingspeak api
  String dataRequest = "GET /update?api_key=" + THINGSPEAK_API + "&field1=" + String(data10) + "&field2=" + String(data25) + "&field3=" + String(data100);

  //enables multiple connections
  //sendCommand("AT+CIPMUX=1",5,"OK"); 
  
  // Establishes communication
  //Command: AT+CIPSTART=0,"TCP","HOST",PORT
  //linkID = 0, for multiple connections
  //type=TCP, host = api.thingspeak.com, 
  sendCommand("AT+CIPSTART=\"TCP\",\""+ THINGSPEAK_HOST +"\","+ THINGSPEAK_PORT,15,"OK");
  
  // Prepares thingspeak for receiving data; sets data request length. +4 for two carriage return and new line, assumedly to create room for new command
  sendCommand("AT+CIPSEND="+String(dataRequest.length()+4),4,">");

  // Actually sends the data
  esp8266.println(dataRequest);
  //countTrueCommand++;

  // Closes communication for linkID = 0
  sendCommand("AT+CIPCLOSE",5,"OK");
}

// ESP+NTP Functions ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
void updateCurrentTime() {
  esp8266.listen();
  epochUnix = getTimeEpochUnix();
  currentHour = (epochUnix % NUMBEROFSECONDSPERDAY) / NUMBEROFSECONDSPERHOUR;
  currentMinute = (epochUnix % NUMBEROFSECONDSPERHOUR) / NUMBEROFSECONDSPERMINUTE;
  currentSecond = epochUnix % NUMBEROFSECONDSPERMINUTE;
}


unsigned long getTimeEpochUnix() {
  unsigned long secsSince1900 = 0UL; //Since we only care about time of day and not date, this isn't really necessary. But we might as well get the full time right anyway

  sendCommand("AT+CIPSTART=\"UDP\",\""+ TIME_HOST +"\","+ TIME_PORT, 15, "OK"); //NOT STRICTLY NECESSARY IF ALREADY CONNECTED?
  sendCommand("AT+CIPSEND=48", 5, "OK");
  
  emptyESP_RX(1000UL);
  esp8266.write((char*)&ntpFirstFourBytes, NTP_PACKET_SIZE);  // Request

  // skip  AT command answer ("Recv 48 bytes\r\n\r\nSEND OK\r\n\r\n+IPD,48:")
  waitForString(":", 1000UL);

  // read the NTP packet, extract the TRANSMIT TIMESTAMP in Seconds from bytes 40,41,42,43
  for (int i = 0; i < NTP_PACKET_SIZE; i++) {
    unsigned long cT = 0;
    unsigned long lT = millis();
    while (!esp8266.available()) {
    }
    int c = esp8266.read();
    if ((i >= 40) && (i < 44)) secsSince1900 = (secsSince1900 << 8) + (unsigned long)((uint8_t)(c & (int)0x00FF));  // Read the integer part of sending time
    else if (i == 44) secsSince1900 += (((uint8_t)c) > SECONDROUNDINGTHRESHOLD ? 1 : 0);
  }

  epochUnix = secsSince1900 - SEVENTYYEARS; // subtract seventy years. Again, since we only care time of day, this isn't strictly necessary.


  sendCommand("AT+CIPCLOSE",5,"OK");
  return epochUnix + UTC_DELTA;
}


void emptyESP_RX(unsigned long duration) {
  unsigned long currentTime;
  currentTime = millis();
  while (millis() - currentTime <= duration) {
    if (esp8266.available() > 0) esp8266.read();
  }
}


boolean waitForString(const char* endMarker, unsigned long duration) {
  int localBufferSize = strlen(endMarker);  // we won't need an \0 at the end
  char localBuffer[localBufferSize];
  int index = 0;
  boolean endMarkerFound = false;
  unsigned long currentTime;

  memset(localBuffer, '\0', localBufferSize);  // clear buffer

  currentTime = millis();
  while (millis() - currentTime <= duration) {
    if (esp8266.available() > 0) {
      if (index == localBufferSize) index = 0;
      localBuffer[index] = (uint8_t)esp8266.read();
      endMarkerFound = true;
      for (int i = 0; i < localBufferSize; i++) {
        if (localBuffer[(index + 1 + i) % localBufferSize] != endMarker[i]) {
          endMarkerFound = false;
          break;
        }
      }
      index++;
    }
    if (endMarkerFound) break;
  }
  return endMarkerFound;
}

// Utility Functions ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
void sendCommand(String command, int maxTime, char readReply[]) {
  // Code for doing a single ESP command
  bool readFound = false;
  int currentCommandTime = 0;

  while(currentCommandTime < (maxTime*1))
  {
    esp8266.println(command);//at+cipsend
    if(esp8266.find(readReply))//ok
    {
      readFound = true;
      break;
    }
  
    currentCommandTime++;
  }

  if (!readFound) {
    Serial.println(String(command) + ": Fail");
  } else {
    Serial.println(String(command) + ": Success");
  }
  //Comment out the else statement for less clutter

}

void displayTime() {
  // Digital clock display of the time, all on the same line. Only displays if time is available, sends message if unavailable.
  Serial.print("Time: ");
  Serial.print(currentHour);
  printDigits(currentMinute);
  printDigits(currentSecond);
  Serial.println();
}


void printDigits(int digits) {
  // utility function for digital clock display: prints preceding colon and leading 0
  Serial.print(":");
  if (digits < 10)
    Serial.print('0');
  Serial.print(digits);
}

Once you have the time from NTP you could use the arduino "TimLib.h" so you could use the arduino time functions for controlling the timing of events.

I guess I'd build in a time out. If it gets stuck waiting for a response from the ESP, then after the time out, print some debug information and repeat the AT command.

Which command is this?

Thanks
-Sean

What is the advantage to using TimeLib.h versus simply updating variables on a minute-by-minute basis?

Thanks
-Sean

If you use the Arduino time library, it mantains a "system time". You can get it to synchronise occasionally and automatically with a time source (you supply the access routine) then you can simply do tests like whenever you choose:

if ( minute() == 54 && second() == 30 ) {
   // do something
}

That means you don't have to keep checking NTP every minute.

Here is something from Red Hat: How to change the NTP polling interval? - Red Hat Customer Portal

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.