Bluetooth Comms. Should I be using Strings? Is there a better way?

I'm developing a simple star tracker module that has bluetooth connection to an app i'm also developing in MIT App Inventor. I'm slowly working through the code parts, ensuring I get all parts working before I move onto the next part.

I'm using an ESP32 and have been able to integrate inputs from a ublox gps module as well as change settings on the ublox over serial interface. I've been able to integrate bluetooth and send the important data through to the basic app.

My next step is to provide a basic level of error checking on the bluetooth interface. I've got a routine ready to go for that and you can see it in the code below. The CRC generation sort of works, but it doesn't and this is where my headline question comes in.

I've set up a basic data structure in strings, using "|" delimiters. But I don't think I fully understand the string structure and now that I'm trying to run the data packet through the CRC part of the code, I realize I may have a hodge-podge of data that isn't very well set up.

I've tried using char's but run into a whole lot of data mis-match issues.

The CRC code is working, but it is only parsing the first value within the string.

Am I on the right track with this and could I be doing it better (i.e simpler/elegant)?

#include <TinyGPSPlus.h>
#include "BluetoothSerial.h" 
#include <ZzzMovingAvg.h>

TinyGPSPlus gps; //GPS module
BluetoothSerial espBT; //Internal Bluetooth
HardwareSerial GPSSerial(2); //GPS serial interface 

 

TinyGPSCustom FixQual(gps, "GNGGA", 6); //Map the GPS Fix details from the GNGGA sentence to FixQual
TinyGPSCustom magDec(gps, "GNRMC", 10); //Map the mangnetic declination angle to magDec
TinyGPSCustom angDec(gps, "GNRMC", 11); // Map the declination direction (east or west) to angDec

unsigned long last = 0UL; //This is the loop timing variable. Set to zero, unsigned long.

//CRC checking
#define UInt16 uint16_t

//Set up 2 layer smoothing of battery readings. This seems to provide the most stable battery readings
ZzzMovingAvg <16> batAvg1; //moving average system for sensor input that's read every loop
ZzzMovingAvg <8, float, float> batAvg2; //moving average system for voltage calculation that's done every 5 seconds

int BatPin = 36; //Battery analog read input pin
float BatLow = 3.1; //Battery low voltage
float BatHigh = 4.19;//Battery high voltage
int BatCells = 6; // The battery monitoring code works on a 3.1(low), 3.7(nom) & 4.2v(high) cell spec. Need to add a cell multiplier for a multi-cell battery.
//const int samples = 100; //Battery sampling - There will be 8 times this many readings. Recommend at least 2



//This is the Ublox Setting Section. These are the base settings to ensure the unit is always set up as required. Change them on on-the-fly later as required.
const char UBLOX_INIT[] PROGMEM = {
  //Set GPS Module to default settings
  0xB5,0x62,0x06,0x09,0x0D,0x00,0xFF,0xFF,0x00,0x00,0x00,0x00,0x00,0x00,0xFF,0xFF,0x00,0x00,0x17,0x2F,0xAE,

  //Set GPS Module to send G*GGA Messages Only
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x00,0x00,0x01,0x00,0x01,0x00,0x00,0x01,0x2B, //GxGGA ON UART1 + USB (On i2C, UART1, USB and SPI by default settings), Lat, Long and Altitude
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x01,0x00,0x00,0x00,0x01,0x00,0x00,0x01,0x2D, //GxGLL USB Only for debugging (On i2C, UART1, USB and SPI by default settings)
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x02,0x00,0x00,0x00,0x01,0x00,0x00,0x02,0x34, //GxGSA USB Only for debugging (On i2C, UART1, USB and SPI by default settings)
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x03,0x00,0x00,0x00,0x01,0x00,0x00,0x03,0x3B, //GxGSV USB Only for debugging (On i2C, UART1, USB and SPI by default settings)
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x04,0x00,0x01,0x00,0x01,0x00,0x00,0x05,0x47, //GxRMC ON UART1 + USB (On i2C, UART1, USB and SPI by default settings) For Declination extraction
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x05,0x00,0x00,0x00,0x01,0x00,0x00,0x05,0x49, //GxVTG USB Only for debugging (On i2C, UART1, USB and SPI by default settings)
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x06,0x00,0x00,0x00,0x00,0x00,0x00,0x05,0x4D, //GxGRS OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x07,0x00,0x00,0x00,0x00,0x00,0x00,0x06,0x54, //GxGST OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x08,0x00,0x00,0x00,0x00,0x00,0x00,0x07,0x5B, //GxZDA OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x09,0x00,0x00,0x00,0x00,0x00,0x00,0x08,0x62, //GxGBS OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x0A,0x00,0x00,0x00,0x00,0x00,0x00,0x09,0x69, //GxDTM OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x0D,0x00,0x00,0x00,0x00,0x00,0x00,0x0C,0x7E, //GxGNS OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x0E,0x00,0x00,0x00,0x00,0x00,0x00,0x0D,0x85, //GxTHS OFF
  0xB5,0x62,0x06,0x01,0x08,0x00,0xF0,0x0F,0x00,0x00,0x00,0x00,0x00,0x00,0x0E,0x8C, //GxVLW OFF

  //Set GPS to Portable Mode
  0xB5,0x62,0x06,0x24,0x24,0x00,0xFF,0xFF,0x00,0x03,0x00,0x00,0x00,0x00,0x10,0x27,0x00,0x00,0x0A,0x00,0xFA,0x00,0xFA,0x00,0x64,0x00,0x5E,0x01,0x00,0x3C,0x00,0x00,0x10,0x27,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xBA,0xB3,

  // Pre-Lock and Lock PPS rate
  //500000us with no GPS Lock + 10395us with Lock
  0xB5,0x62,0x06,0x31,0x20,0x00,0x01,0x01,0x00,0x00,0x32,0x00,0x00,0x00,0x20,0xA1,0x07,0x00,0x9B,0x28,0x00,0x00,0x9A,0x99,0x99,0x19,0x9A,0x99,0x99,0x19,0x00,0x00,0x00,0x00,0x67,0x00,0x00,0x00,0x47,0x23, 

  // Rate at which GPS module sends data
  //0xB5,0x62,0x06,0x08,0x06,0x00,0x64,0x00,0x01,0x00,0x01,0x00,0x7A,0x12, //10Hz (100ms)
  //0xB5,0x62,0x06,0x08,0x06,0x00,0xC8,0x00,0x01,0x00,0x01,0x00,0xDE,0x6A, //5Hz (200ms)
  //0xB5,0x62,0x06,0x08,0x06,0x00,0xE8,0x03,0x01,0x00,0x01,0x00,0x01,0x39, //1Hz (1000ms)
  //0xB5,0x62,0x06,0x08,0x06,0x00,0xD0,0x07,0x01,0x00,0x01,0x00,0xED,0xBD, //0.5Hz (2000ms)
  0xB5,0x62,0x06,0x08,0x06,0x00,0x88,0x13,0x01,0x00,0x01,0x00,0xB1,0x49, //0.2Hz (5000ms)
};



void setup() 
  {
  Serial.begin(9600);
  GPSSerial.begin(9600,SERIAL_8N1,16,17); //This is two-way communication. If I just want to receive data from GPS, use: GPSSerial.begin(9600,SERIAL_8N1,16,-1)
  espBT.begin("StarTrackerBT");  

 // send the initial configuration data in UBX protocol
  for(int i = 0; i < sizeof(UBLOX_INIT); i++) {                        
    GPSSerial.write( pgm_read_byte(UBLOX_INIT+i) );
    delay(5); // simulating a 38400baud pace (or less), otherwise commands are not accepted by the device.
    }
  }



//CRC Error checking routine
UInt16 ModRTU_CRC(char * buf, int len)
{
  UInt16 crc = 0xFFFF;
  for (int pos = 0; pos < len; pos++) {
    crc ^= (UInt16)buf[pos];          // XOR byte into least sig. byte of crc
    for (int i = 8; i != 0; i--) {    // Loop over each bit
      if ((crc & 0x0001) != 0) {      // If the LSB is set
        crc >>= 1;                    // Shift right and XOR 0xA001
        crc ^= 0xA001;
      }
      else                            // Else LSB is not set
        crc >>= 1;                    // Just shift right
    }
  }
  // Note, this number has low and high bytes swapped, so use it accordingly (or swap bytes)
  return crc;  
}


void loop() 
  {
  //Read battery in every loop
  int avgBattRead = batAvg1.add(analogRead(BatPin));

  //Read GPS serial interface in every loop
  while (GPSSerial.available() > 0) 
  gps.encode(GPSSerial.read());

  //Set up loop to send monitoring data every 1 second.
  if (millis() - last > 5000)
  {

    //Check Battery Level
    uint8_t percentage = 100;
    float voltage = (avgBattRead / 4096.0 * 6.595);    //  ESP32 pin 36(SP) with 100K+9K1/0.1uF voltage divider added. 
    float aveVoltage = batAvg2.add(voltage); //Average-out the voltage conversions - this is the secon dlayer of smoothing
    percentage = 3451.6223*pow(aveVoltage, 6) - 74273.6875*pow(aveVoltage, 5) + 663793.0211*pow(aveVoltage, 4) - 3153807.5416*pow(aveVoltage, 3) + 8402037.133*pow(aveVoltage, 2) - 11900941.1029*aveVoltage + 7002339.4288;
    if (aveVoltage > (BatHigh)) percentage = 100;
    else if (aveVoltage <= (BatLow)) percentage = 0;

    //Battery Dataset
    String d1 = String(aveVoltage*BatCells);
    String d2 = String(percentage);

       
    //GPS Dataset
    String d3 = String(gps.location.lat(),6); //3 - Latitude
    String d4 = String(gps.location.lng(),6); //4 - Longitude
    String d5 = String(gps.altitude.meters(),2); //5 - Altitude
    String d6 = String(magDec.value(),4); //6 - Declination
    String d7 = String(angDec.value(),1); //7 - Declination direction
    String d8 = String(gps.satellites.value()); //8 - Number of Satellites
    String d9 = String(FixQual.value()); //9 - Quality of fix



    //Interface Dataset
    //String d10 = String(button 1 condition); // 
    //String d11 = String(button 2 condition); // 
    //String d12 = String(button 3 condition); // 
    //String d13 = String(button 4 condition); // 
    //String d14 = String(button 5 condition); // 
    //String d15 = String(button 6 condition); // 
    //String d16 = String(button 7 condition); // 
    //String d17 = String(button 8 condition); // 
    
    int dataBTlen = 9+8; // Number of items in datasets above + "|" delimieters
    String dataBT[dataBTlen] = {d1, "|",d2, "|",d3, "|",d4, "|",d5, "|",d6, "|",d7, "|", d8, "|", d9};//cocenate all the data with "|" delimiter
   
 

    //Calculate CRC16 of the data string
   //dataBTtxt = dataBT;
   char *t = (char *)dataBT;
    

    //Send off all the data to bluetooth
    for (int i=0;i<dataBTlen;i++) espBT.print(dataBT[i]);
    espBT.println();



    //SERIAL PRINT SECTION

    //CRC CHecksum for current Bluetooth Data Transfer
    Serial.print("Bluetooth data string: ");
    for (int i=0;i<dataBTlen;i++) Serial.print(dataBT[i]);
    Serial.println();
    Serial.print("CRC16 code: ");
    Serial.println(ModRTU_CRC(t,strlen(t)),HEX);

    //Battery Status
    Serial.println("Battery Voltage = " + d1 + "v, Percentage = " + d2 +"%");

    //Location information
    Serial.print("Latitutde=");
    Serial.print(gps.location.lat(), 6);
    Serial.print(" Longitude=");
    Serial.print(gps.location.lng(), 6);
    Serial.print(" Altitude=");
    Serial.print(gps.altitude.meters());
    Serial.print(" Declination angle=");
    Serial.print(magDec.value());
    Serial.println(angDec.value());
    
    //Diagnostic Information
    Serial.print(F("DIAGS      Chars="));
    Serial.print(gps.charsProcessed());
    Serial.print(F(" Sentences-with-Fix="));
    Serial.print(gps.sentencesWithFix());
    Serial.print(F(" Failed-checksum="));
    Serial.print(gps.failedChecksum());
    Serial.print(F(" Passed-checksum="));
    Serial.println(gps.passedChecksum());

    //Quality of GPS Positioning
    Serial.print(F("Number of Satellites = "));
    Serial.print(gps.satellites.value());
    Serial.print(F(", Age of Fix = "));
    Serial.print(gps.satellites.age());
    Serial.print(" Quality of fix = ");
    Serial.print(FixQual.value());
    Serial.println(", (0=invalid, 1=GPS Fix (Standard Position Service), 2=DGPS (Differential GPS)");
   
    
    if (gps.charsProcessed() < 10)
      Serial.println(F("WARNING: No GPS data.  Check wiring."));
  
    last = millis();
    Serial.println();

  }
   
  }



CRC can be done with hardware.

This is not the way :

int dataBTlen = 9+8; // Number of items in datasets above + "|" delimieters
    String dataBT[dataBTlen] = {d1, "|",d2, "|",d3, "|",d4, "|",d5, "|",d6, "|",d7, "|", d8, "|", d9};//cocenate all the data with "|" delimiter

Like that you store the pointers to the Strings, i don't know how that will fare, but i am pretty sure you could just do

 String dataBT = d1 + "|" + d2 + "|" + d3 + "|" + d4 +  "|" + d5 + "|" + d6 + "|" + d7 + "|" + d8 + "|" + d9;

To get the desired result.
You could of course try with c-strings, but the memory on an ESP is quite sufficient and as long as the declared Strings are declared locally and no other lasting object on the heap gets modified (expanded) in the mean time there is no chance fragmentation.

once you have solved the concatenation with the String (not great, lots of transitory Strings being created) you'll need to solve for this

you can't just refactor the dataBT into a char * pointer. You could use dataBT.c_str() to get the const char * underlying buffer.

String dataBT = d1 + "|" + d2 + "|" + d3 + "|" + d4 +  "|" + d5 + "|" + d6 + "|" + d7 + "|" + d8 + "|" + d9;
const char * t = dataBT.c_str();

Thank you @Deva_Rishi and @J-M-L for the input.

I think I now see I was creating a string array and mixing data types all over the shop, but calling them all strings.

I've implemented your code suggestions and now started understanding what each of the datatypes is starting with the GPS outputs. i.e:

    //GPS Dataset
    double d3 = gps.location.lat(); //3 - Latitude
    double d4 = gps.location.lng(); //4 - Longitude
    double d5 = gps.altitude.meters(); //5 - Altitude
    const char* d6 = magDec.value(); //6 - Declination
    const char* d7 = angDec.value(); //7 - Declination direction
    uint32_t d8 =  gps.satellites.value(); //8 - Number of Satellites
    const char* d9 = FixQual.value(); //9 - Quality of fix

That's all working well.

However, when it comes to my "Battery Dataset", these sinppets create a problem:

int BatCells = 6;

//Check Battery Level
    uint8_t percentage = 100;
    float voltage = (avgBattRead / 4096.0 * 6.595);    //  ESP32 pin 36(SP) with 100K+9K1/0.1uF voltage divider added. 
    float aveVoltage = batAvg2.add(voltage); //Average-out the voltage conversions - this is the secon dlayer of smoothing
    percentage = 3451.6223*pow(aveVoltage, 6) - 74273.6875*pow(aveVoltage, 5) + 663793.0211*pow(aveVoltage, 4) - 3153807.5416*pow(aveVoltage, 3) + 8402037.133*pow(aveVoltage, 2) - 11900941.1029*aveVoltage + 7002339.4288;
    if (aveVoltage > (BatHigh)) percentage = 100;
    else if (aveVoltage <= (BatLow)) percentage = 0;


float d1 = aveVoltage*BatCells;

And then when I come to concatenate:

String dataBT = d1+"|"+d2+"|"+d3+"|"+d4+"|"+d5+"|"+d6+"|"+d7+"|"+d8+"|"+d9;//cocenate all the data with "|" delimiter

I get the error:

Compilation error: invalid operands of types 'float' and 'const char [2]' to binary 'operator+'

I don't quite understand how variables d3-d9 can be a mix of doubles const char's uint32_t and they all join, but when it comes to the float d1, I get this error.

That’s because the float is at the start of the chain and the + becomes the mathematical operator between d1 and “|”

Use


String dataBT = String(d1);
dataBT  += "|";
dataBT  +=  String(d2) ;
dataBT  += "|";
...  // I let you fill that in
dataBT  += "|";
dataBT  +=  String(d9) ;

+= will be more efficient than just + in between all the elements

Thanks @J-M-L ,

Should all of the data values be added as strings, or should I simply add each of the d2-d9 to the new string, thus:

    String dataBT = String(d1);
    dataBT  += "|";
    dataBT  +=  d2 ;
    dataBT  += "|";
    dataBT  +=  d3 ;
    dataBT  += "|";
    dataBT  +=  d4 ;
    dataBT  += "|";
    dataBT  +=  d5 ;
    dataBT  += "|";
    dataBT  +=  d6 ;
    dataBT  += "|";
    dataBT  +=  d7 ;
    dataBT  += "|";
    dataBT  +=  d8 ;
    dataBT  += "|";
    dataBT  +=  d9 ;

or set up a for int i=2 loop if the data structure becomes much longer.

Make the life of the compiler easy and String() them all
If they were in an array then a loop would help

1 Like

Thank you both for contributing to this solution.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.