Best practice for splitting strings

Projects Programming
1

I have an ESP8266 Node MCU board that connects to wifi, downloads a weather API and parses the data. I can then extract any data I need from the parsed string and send it to Thingspeak. I also send it to an UNO via i2c.

I can send the data in any format such as colon delimited-
12.34 : 99 : some_text : 56.7
or each value can have a 'header'-
a 12.34 : b 99 : c some_text : d 56.7

What I would like to do is extract the sent data on the UNO and assign each value to a variable that I can manipulate as required and then display on an LCD .

I have been reading a lot on Google but it seems a total mine field of conflicting information. (Some say use Strings others say NEVER use Strings) and so on.

Is there a 'best practice' way of doing this?

2

Start here

Come back if it doesn't fix you right up.

I vote no Capital S Strings until

you have fully mastered real small s strings by which time

you will never ever need Capital S Strings.

Ducking for cover now.

HTH

a7

3

On an ESP8266 I would use JSON. The ArduinoJson.org "Assistant" can write the Serialize and Deserialize code for you.

4

Yes, this is what I have done.
The ESP side of my project is complete and working as I want. It's the UNO side that I am stuck on where manipulating the data is required

5

With I2C, I would not be sending as ASCII text. Is the "some text" a specific length, or a known maximum length?

6

The text part can vary in length ('overcast', 'rain', 'thunder' etc)

7

I looked and there is JSON for UNO, could you use that? Looks plausible and only a few thousand bytes on your sketch...

a7

8

I have just tried your example number 5 from your first reply and I think that is all I need.
I sent a string of various numbers and text and split them up and manipulated them and the results were good.
I do have to send the data in 'packets' as it exceeds the 32byte limit for Wire.h over i2c
I will see if different start and end characters help me solve this.

9

I can now send the required data by splitting it into sections between 'Wire.beginTransmission' and 'Wire.endTransmission' with the end markers'< and>' added at the beginning and the end of the whole block of data.
I am currently having trouble with certain data.
One of the numbers is a ten digit whole number (not negative or decimal).
I only receive the first 5 digits.
I have tried 'int', 'long' etc but still get the same result.

10

I may have spotted a way to make your problem disappear ...

Don't use the Uno.

What is the Uno doing? Why can't the esp do it instead? It is a significantly more powerful chip. Of course, it has fewer pins out-of-the-box, but there are much better ways of adding more pins than connecting it to another MCU. Using more than one MCU in any project, or trying to, is a mistake beginners often make. It always makes things more complex than they should be because of the problems of communicating between the two MCU, as you are discovering.

11

I do have the ESP doing everything. Even displaying on two small Oled displays
I need the Uno to display on a TFT

12

I challenge that assumption. I suspect the esp can do a far better job of displaying on the TFT, the Uno is only getting in the way and making things difficult and complicated.

Post a link to the specs of the TFT and see what the forum can suggest.

13

I have an Adafruit TFT that sits on top of the Uno, I have already adapted it so that I can use the SDA and SCL pins (The TFT uses pin 4 as a reset)

I have everything working perfectly except for the problem with the 10 digit output

14

How are you converting the ASCII to long int? atoi is for a 16-bit integer, atol is for long int

As I said before, with I2C much easier to just send as a long int and drop the conversion to ASCII and back.

15

I am simply extracting the required data from a parsed string and then breaking it up and sending it to the Uno.
In serial.print the data would read as follows:

<
cloudy
801
0.25
1614716189
21.2

On the Uno I receive:
<
cloudy
801
0.25
16147
0.00

16

Time to tell us more by… posting the code or a reduced sketch that shows the problem.

Sounds like you almost there, prolly something dumb or simple or both. :expressionless:

a7

17

I have a line in the ESP code as follows:

const size_t bufferSize =  JSON_ARRAY_SIZE(1)  + 2 * JSON_OBJECT_SIZE(4) +  JSON_OBJECT_SIZE(10) +6 * JSON_OBJECT_SIZE(10);

Whatever I change this to makes no difference to the output of the json string.
What does it do?
Is there something I need to add/change?

18

This is the Uno code:

#include <Wire.h>
const byte numChars = 32;
char receivedChars[numChars];
char tempChars[numChars];        // temporary array for use when parsing

// variables to hold the parsed data

int weather_id = 0;
char weather_description[numChars] = {0};
float moon_phase = 0.0;
int sunrise = 0;
int sunset = 0;
float c_temp;
float c_humi = 0.0;
float c_dp = 0.0;
float temp = 0.0;
float humi = 0.0;

boolean newData = false;
int newdata;
//============

void setup() {

  Wire.begin(8);
  Wire.onReceive(recvWithStartEndMarkers);

  Serial.begin(9600);

}

//============

void loop() {
  // recvWithStartEndMarkers();
  if (newData == true) {
    strcpy(tempChars, receivedChars);
    // this temporary copy is necessary to protect the original data
    //   because strtok() used in parseData() replaces the commas with \0
    parseData();
    showParsedData();
    newData = false;
  }
  Serial.print(".");
  delay(1000);
}

//============

void recvWithStartEndMarkers() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = '<';
  char endMarker = '>';
  char rc;

  while (Wire.available() > 0 && newData == false) {
    rc = Wire.read();

    if (recvInProgress == true) {
      if (rc != endMarker) {
        receivedChars[ndx] = rc;
        ndx++;
        if (ndx >= numChars) {
          ndx = numChars - 1;
        }
      }
      else {
        receivedChars[ndx] = '\0'; // terminate the string
        recvInProgress = false;
        ndx = 0;
        newData = true;
      }
    }

    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
}

//============

void parseData() {      // split the data into its parts

  char * strtokIndx; // this is used by strtok() as an index

  strtokIndx = strtok(tempChars, ",");     // get the first part - the string
  strcpy(weather_description, strtokIndx); // copy it to messageFromPC

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  weather_id = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ",");
  moon_phase = atof(strtokIndx);     // convert this part to a float

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  sunrise = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  sunset = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ",");
  c_temp = atof(strtokIndx);     // convert this part to a float

  strtokIndx = strtok(NULL, ",");
  c_humi = atof(strtokIndx);     // convert this part to a float

  strtokIndx = strtok(NULL, ",");
  c_dp = atof(strtokIndx);     // convert this part to a float

  strtokIndx = strtok(NULL, ",");
  temp = atof(strtokIndx);     // convert this part to a float

  strtokIndx = strtok(NULL, ",");
  humi = atof(strtokIndx);     // convert this part to a float

}

//============

void showParsedData() {
  Serial.println();
  Serial.print("Weather Description ");
  Serial.println(weather_description);

  Serial.print("Weather id ");
  Serial.println(weather_id);



  Serial.print("Moon Phase ");
  Serial.println(moon_phase);

  Serial.print("Sunrise ");
  Serial.println(sunrise);

  Serial.print("Sunset ");
  Serial.println(sunset);

  Serial.print("Current Temperature ");
  Serial.println(c_temp);

  Serial.print("Current Humidity ");
  Serial.println(c_humi);

  Serial.print("Current Dewpoint ");
  Serial.println(c_dp);

  Serial.print("Indoor Temperature ");
  Serial.println(temp);

  Serial.print("Indoor Humidity ");
  Serial.println(humi);


  Serial.println();
}
19

And here is the (vert heavily edited) ESP code:

#include <Wire.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClient.h>
//#include <Arduino_JSON.h>
#include <ArduinoJson.h>
#include <TimeLib.h>
#include <DHT.h>
#include <SPI.h>
#include <Wire.h>
#define DHTPIN 2
//unsigned long t_unix_date1;

WiFiClient client;
const char* ssid = "VM1654567";
const char* password = "jtzh3MkYgbh4";
DHT dht(DHTPIN, DHT22);

//String openWeatherMapApiKey = "51ddde7c19ccef007569f7922c24887d";
String apiKey = "xxxxxx";
const char server[] = "api.openweathermap.org";

String jsonBuffer;

int weather_id;
String weather_description;
float moon_phase;
long int sunrise;
long int sunset;
float c_temp;
float c_humi;
float c_dp;

String text;
int jsonend = 0;
boolean startJson = false;
#define JSON_BUFF_DIMENSION 2500


void setup() {
  Serial.begin(9600);
  text.reserve(JSON_BUFF_DIMENSION);
  Wire.begin(5, 4);

  WiFi.begin(ssid, password);
  Serial.println("Connecting");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to WiFi network with IP Address: ");
  Serial.println(WiFi.localIP());

}
void loop() {

  makehttpRequest();
//other code here creating a delay.(connect to Thingspeak etc)
}

}
void makehttpRequest() {

  client.stop();

  if (client.connect(server, 80)) {
    Serial.println("HTTP connecting...");
    // send the HTTP PUT request:
    //  client.println("GET /data/2.5/forecast?q=" + nameOfCity + "&APPID=" + apiKey + "&mode=json&units=metric&cnt=2 HTTP/1.1");
    client.println("GET /data/2.5/onecall?lat=52.4976&lon=-2.1689&exclude=minutely,hourly,alerts&units=metric&appid=51ddde7c19ccef007569f7922c24887d");

    //  String serverPath = "http://api.openweathermap.org/data/2.5/onecall?lat=52.4976&lon=-2.1689&exclude=minutely,hourly,alerts&units=metric&appid=xxxxxxx";

    client.println("Host: api.openweathermap.org");
    client.println("User-Agent: ArduinoWiFi/1.1");
    client.println("Connection: close");
    client.println();
    Serial.println("Success ");
    unsigned long timeout = millis();
    while (client.available() == 0) {
      if (millis() - timeout > 5000) {
        Serial.println(">>> Client Timeout !");
        client.stop();
        return;
      }
    }

    char c = 0;
    while (client.available()) {
      c = client.read();

      if (c == '{') {
        startJson = true;         // set startJson true to indicate json message has started
        jsonend++;
        //      Serial.println("1");
      }
      if (c == '}') {
        jsonend--;
        delay(10);
      }
      if (startJson == true) {
        text += c;
        //     Serial.print("*");
      }
      // if jsonend = 0 then we have have received equal number of curly braces
      if (jsonend == 0 && startJson == true) {
        parseJson(text.c_str());  // parse c string text in parseJson function
        text = "";                // clear text string for the next time
        startJson = false;        // set startJson to false to indicate that a new message has not yet started

      }
    }
  }
  else {
    // if no connction was made:
    Serial.println("connection failed");
    return;
  }
}


void parseJson(const char * jsonString) {

   //StaticJsonBuffer<5000> jsonBuffer;
  const size_t bufferSize =  JSON_ARRAY_SIZE(1)  + 2 * JSON_OBJECT_SIZE(4) +  JSON_OBJECT_SIZE(10) +6 * JSON_OBJECT_SIZE(10);
 DynamicJsonBuffer jsonBuffer(bufferSize);

  // FIND FIELDS IN JSON TREE
  JsonObject& root = jsonBuffer.parseObject(jsonString);
  if (!root.success()) {
    Serial.println("parseObject() failed");
    return;
  }

  JsonArray& list = root["list"];
  JsonObject& nowT = list[0];
  JsonObject& later = list[1];

  int weather_id = root["current"]["weather"][0]["id"];
  String weather_description = root["current"]["weather"][0]["description"];
  float moon_phase = root["daily"][0]["moon_phase"];
  int sunrise = root["current"]["sunrise"];
  long int sunset = root["current"]["sunset"];
  float c_temp = root ["current"]["temp"];
  float c_humi = root["current"]["humidity"];
  float c_dp = root["current"]["dew_point"];

  Serial.print("Weather Description ");
  Serial.println(weather_description);
  Serial.print("Weather id ");
  Serial.println(weather_id);
  Serial.print("Moon Phase ");
  Serial.println(moon_phase);
  Serial.print("Sunrise ");
  Serial.println(sunrise);
  Serial.print("Sunset ");
  Serial.println(sunset);
  Serial.print("Current Temperature ");
  Serial.println(c_temp);
  Serial.print("Current Humidity ");
  Serial.println(c_humi);
  Serial.print("Current Dewpoint ");
  Serial.println(c_dp);
  Serial.print("Indoor Temperature ");
  Serial.println(temp);
  Serial.print("Indoor Humidity ");
  Serial.println(humi);

  Wire.beginTransmission(8);
  Wire.print("<");
  Wire.print(weather_description);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(weather_id);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(moon_phase);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(sunrise);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(sunset);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(c_temp);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(c_humi);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(c_dp);
  Wire.print(",");
  Wire.endTransmission();
  Wire.beginTransmission(8);
  Wire.print(temp);
  Wire.print(",");
  Wire.print(humi);
  Wire.print(">");
  Wire.endTransmission();


}
20

What line of the UNO code is parsing and/or printing a value incorrectly?

You make it sound like the packet is malformed at the sender side already.

It sounds like you are able to examine the packet on the sender side and find it incorrect?

That constant that sizes a buffer in #17, what is its value? Can you just make it a large number directly (no JSON calls), large enough for anything that might go in there, fix it later to be “just big enough”.

Also in #15, it looks like the float direct aftert the bad integer is different, is that another fault of the process?

a7