How to extract HEX from this String object

ninja2 · January 4, 2025, 9:18am

I have a a Victron battery monitor that acts as a Bluetooth Low Energy (BLE) peripheral named "BATTbank_SmartShunt". It advertises the usual data for BLE, including "Extra Manufacturer Data". As the data is advertised it can be read without connecting. Some of the data is also encrypted.

This is what the advertised data looks like when I use the excellent BLE sniffer app "nRF Connect".

The manufacturer data is the group of approx. 30 HEX bytes starting 0x100289A302BC ... etc.
Once I get hold of these bytes (!) I will be able parse them one by one, then decrypt to extract info like battery volts, amps etc.

I have made a sketch to receive the advertised data on my ESP32. Unfortunately I have to use the Bluetooth LE library included in ESP32 package and it returns the manufacturer data in Arduino String format. I always avoid using String ! But I have no choice, it's used throughout the lib..

My sketch is derived from the "Client" example provided with the BLE library. It's working in part, producing this output:

This is the relevant part of my sketch

#include <Streaming.h>

class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks {
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    if (advertisedDevice.getName() == "BATTbank_SmartShunt") {
      BLEDevice::getScan()->stop();
      myDevice = new BLEAdvertisedDevice(advertisedDevice);
      Serial << "Found: (" 
            << advertisedDevice.getManufacturerData().length() << ") "
            << advertisedDevice.getName().c_str()              << " [" 
            << advertisedDevice.getManufacturerData()          << "]\n";
      }
    }  // onResult
  };  // MyAdvertisedDeviceCallbacks

If I instead use this above

advertisedDevice.getManufacturerData().c_str()

the result is the same.

This snippet is an extract from BLEAdvertisedDevice.cpp in the library:

/**
* @brief Get the manufacturer data.
* @return The manufacturer data of the advertised device.
*/
String BLEAdvertisedDevice::getManufacturerData() {
 return m_manufacturerData;
}  // getManufacturerData

I've tried using .toCharArray() and .toInt() and even _HEX, but so far I have not extracted the HEX data I need.

Any and all coding suggestions welcomed ...
TIA

UKHeliBob · January 4, 2025, 9:45am

Please post an example of the String that is received

horace · January 4, 2025, 9:59am

there should be no problem using String on a powerful microcontroller such as an ESP32 - it is recommended to avoid its use on low power micros such as the UNO, Nano, etc as it can fragment memory

your explanation of the format of the received data is vague - assuming it is "0x100289A302BC" you can use sscanf() to parse the hex values by specifying the field width of each value, e.g. assuming the data is integer

void setup() {
  String data = "0x100289A302BC";
  Serial.begin(115200);
  delay(2000);
  int d1, d2, d3;
  if (sscanf(data.c_str(), "0x%2x%4x%6x", &d1, &d2, &d3) == 3)
    Serial.printf("d1 %x d2 %x d3 %x\n", d1, d2, d3);
}

void loop() {}

serial monitor displays

d1 10 d2 289 d3 a302bc

ninja2 · January 4, 2025, 12:19pm

thanks for the inputs. I tried your suggestion as follows:

class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks {
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    if (advertisedDevice.getName() == "BATTbank_SmartShunt") {
      BLEDevice::getScan()->stop();
      myDevice = new BLEAdvertisedDevice(advertisedDevice);
      Serial << "Found: (" 
            << advertisedDevice.getManufacturerData().length() << ") "
            << advertisedDevice.getName().c_str() << " [" 
            << advertisedDevice.getManufacturerData().c_str() << "]\n";
      int d1, d2, d3;
      if (sscanf(advertisedDevice.getManufacturerData().c_str(), "0x%2x%4x%6x", &d1, &d2, &d3) == 3)
        Serial.printf("d1 %x d2 %x d3 %x\n", d1, d2, d3);
      else Serial << " fail\n";
      }
    }  // onResult
  };  // MyAdvertisedDeviceCallbacks

but it always branches to "fail" after if(sscanf
(i.e. no output by Serial.printf)

TBH I'm at the limits of my programming skills. (I rarely work inside a class and never use sscanf). So a few questions if I may:

Is it necessary for there to be a d1,d2,d3,d4,d5 etc matching every part of the input String data?
If that is important here is the format of the data that appears after
Victron Energy BV <0x02E1> in the first image/screenshot in my initial post.

10 Manufacturer Data Record Type (always (0x10)
0289 Model Id for Smartshunt Battery Monitor
A3 "read out type" probably always 0xA3 for this config
02 Record Type: 0x02 = Battery Monitor (ref B)

the data above remains constant, whereas
the following data is updated continually, so below are example data only:

5D97 Nonce/Counter, the "salt" used in decryption (little endian)

95 Constant, so far (should match first byte of Encryption Key for this device)

F874601C3F869B6A196258EB22EDD314 Encyrpted data (16 bytes, always ending in 0x14?)

For a total of 24 bytes + terminator = 25 bytes
which is correct, and that is what we see reported by advertisedDevice.getManufacturerData().length()

Maybe the data isn't String type after all? I assumed this based on what I found in BLEAdvertisedDevice.cpp in the library.

Not sure where to go from here ...?

horace · January 4, 2025, 12:26pm

as @UKHeliBob asked for in post 2 please post an example of the String that is received
e.g. what does the following display on the Serial console?

      Serial << "Found: (" 
            << advertisedDevice.getManufacturerData().length() << ") "
            << advertisedDevice.getName().c_str() << " [" 
            << advertisedDevice.getManufacturerData().c_str() << "]\n";

ninja2 · January 4, 2025, 12:33pm

I thought I provided that already ... the second attachment (a JPG) in my initial post shows the Serial Monitor output to exactly the code you've highlighted ?

namely this line
Found: (25) BATTbank_SmartShunt [gobbeldy-goop]

The manufacturer data is the gobbeldy-goop between the square brackets.
I hope this is what you're after ... ?

Here's the output when I use the sscanf code shown above, with "fail" branch

horace · January 4, 2025, 12:42pm

it is probably in raw binary - try printing it's values

for (int i = 0; i < advertisedDevice.getManufacturerData.length(); i++)
    Serial.print(advertisedDevice.getManufacturerData.c_str()[i], HEX);

ninja2 · January 4, 2025, 12:51pm

major success
This code:

class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks {
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    if (advertisedDevice.getName() == "BATTbank_SmartShunt") {
      BLEDevice::getScan()->stop();
      myDevice = new BLEAdvertisedDevice(advertisedDevice);
      Serial << "Found: (" 
            << advertisedDevice.getManufacturerData().length() << ") "
            << advertisedDevice.getName().c_str() << " [" 
            << advertisedDevice.getManufacturerData().c_str() << "]\n";
    Serial << "Raw HEX: ";            
    for (int i = 0; i < advertisedDevice.getManufacturerData().length(); i++)
    Serial.print(advertisedDevice.getManufacturerData().c_str()[i],HEX);
    Serial << '\n';            
    }
  }  // onResult
};  // MyAdvertisedDeviceCallbacks

Produced this:

now I just have to figure out how to deal with the raw binary.

thank you horace !

david_2018 · January 4, 2025, 12:54pm

Be careful with that, if the hex value is 0x00 through 0x0F, the leading zero will not be printed.
You can make this obvious by printing a space between each byte of data.

ninja2 · January 4, 2025, 1:00pm

good point
in fact too good to ignore, so I tried this

    Serial << "Raw HEX: ";            
    for (int i = 0; i < advertisedDevice.getManufacturerData().length(); i++){
      Serial.print(advertisedDevice.getManufacturerData().c_str()[i],HEX);
      Serial << " ";}
    Serial << '\n';

giving this

much better

sonofcy · January 4, 2025, 1:09pm

I also have a shunt, if I want to know that data, I just look at the meter.

ninja2 · January 4, 2025, 1:11pm

I can do the same, also with the VictronConnect app,
but I'm combining Smartshunt data with other data on another output device ...
it's a hobby, don't ask

horace · January 4, 2025, 1:12pm

by knowing the offsets into the array you can pick out the data, e.g.

int modelID=advertisedDevice.getManufacturerData().c_str()[3]<<8 | advertisedDevice.getManufacturerData().c_str()[4];

should give a HEX value 0x0289

ninja2 · January 4, 2025, 1:14pm

excellent thanks. I'll definitely give that a go tomorrow (nearly midnight here ...)

system · July 3, 2025, 1:15pm

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
ESP32 Arduino BLE how to read hex data Home Automation	13	18260	December 7, 2021
Need help deciphering hex values returned by blue tooth module Programming	3	592	May 5, 2021
Get a string from a incoming Bluetooth data Programming	5	1152	May 5, 2021
Bluetooth + HEX + TEXT Programming	4	1636	May 5, 2021
ESP32 BLE how to convert Data to Hex/DEC Programming	6	2638	September 29, 2022

How to extract HEX from this String object

Related topics