Heltec WiFi LoRa 32 (V3) - Encode/decode packets to binary (make message shorter)

Hello,
I'm using Heltec WiFi LoRa32 boards (ESP32-S3 + SX1262).

Could you please help me how to encode data e.g. from temperature and humidity sensor into binary or hexadecimal form - to make the LoRa packet length shorter? Similar to what LoRaWAN does.

For example I have two values:

float temperature = 21.56;
float humidity = 76.24;

I use the official Heltec library for LoRa radio (Heltec_ESP32/src/radio/radio.c at master · HelTecAutomation/Heltec_ESP32 · GitHub), I only transfer data via LoRa (not LoRaWAN), so it is necessary to make it work with the library function Radio.Send().

Radio.Send( (uint8_t *)txpacket, strlen(txpacket) ); //send the package out

I found this example: Arduino Cloud and this tutorial: Working with Bytes | The Things Network

I'm confused in this, so I would appreciate a part of code example compatible with Heltec library for encoding and decoding compatible with Heltec library functions.

Thank you

To send two float values in binary, it is convenient to put them in an array, as follows.

float values[2]={temperature, humidity};
Radio.Send( (uint8_t *)values, sizeof(values) ); //send the package out

However, for portability (there is more than one representation for binary float values), I would make two integers, after multiplying by 100 to preserve two decimal fraction digits and send those:

int values[2]={(int)100*temperature,(int)100*humidity};
Radio.Send( (uint8_t *)values, sizeof(values) ); //send the package out

Note: array names are pointers, so the (uint8_t *) tells the compiler to treat the array name as a pointer to a byte stream.

To save even more space, declare the integer array as 16 bits, as two bytes will store reasonable values of temperature or humidity, even when multiplied by 100.
int16_t values[2]={(int16_t)100*temperature,(int16_t)100*humidity};

Ok fine, and how could be the transmitted values decoded on the receiver's side? Is there any convenient way to decode the values if I know that, for example, each value is a decimal number of length 2 bytes?

Something more convenient than this I mean:

Thanks

If transmitting two floats, here is one approach

float values[2]={0};
memcpy(values,payload,sizeof(values));
Serial.print(values[0]);
Serial.print(values[1]);

Ok, and for case when I don't know how many values will the receiver get? (I only know that every value is a decimal number of length 2 bytes).

If you send 8 bytes, the receiver will receive 8 bytes. If it doesn't, the message is in error.

Although you do know how many bytes the receiver receives. The Radio.receive() (or .read() whichever is applicable) returns the message length in bytes.

If you weren't aware of that, now would be a good time to review the library documentation and examples.

If you use the callback receiver function, study this example carefully:

void OnRxDone( uint8_t *payload, uint16_t size, int16_t rssi, int8_t snr )
{
    rssi=rssi;
    rxSize=size;
    memcpy(rxpacket, payload, size );
    rxpacket[size]='\0';
    Radio.Sleep( );
    Serial.printf("\r\nreceived packet \"%s\" with rssi %d , length %d\r\n",rxpacket,rssi,rxSize);
    lora_idle = true;
}

Ok, since each node measures a different number of quantities, I need the receiver to be able to deal with it and not have to receive a fixed length from all nodes.

It's actually good that the Radio.receive() function can also know the length of the received string and therefore, if I understand correctly, the number of bytes.
If, for example, I received a message with a length of 8 bytes, I know that I received 4 float values, if I understand correctly.

But isn't this code only for one float value (2 bytes)?

float values[2]={0};
memcpy(values,payload,sizeof(values));
Serial.print(values[0]);
Serial.print(values[1]);

I would always need to connect the 2 bytes together.

Thanks

One float value occupies four bytes. I was following your temperature & humidity float example, so eight bytes total.

On the ESP32, an "int" variable also occupies four bytes. Hence int16_t (two byte integer) is very useful for saving memory and space.

Aha, oh sorry.
I was a bit confused by this: Arduino Editor

There they just use int16_t and then send the (originally float) values in 2 bytes.

Ok. So my new exapmle is:
The first node measures 4 decimal values -> 8 int16_t bytes.
The second node measures 6 decimal values -> 12 int16_t bytes.

I receive LoRa messages from both nodes with one receiver. So, is there any convenient way, how to split the received payloads to individual values regardless of the length of the received payload?

Sorry, the question doesn't make much sense.

You send data to the other radio in some predetermined format, and the code for that radio must interpret the received message correctly. There are uncountably many ways to accomplish that basic task, even for something as simple as two measurements.

for an example of transmitting data between LoRa nodes using structures have a look at post need-advice-on-a-home-project
the example uses the Arduino LoRa library but the technique works with other libraries

There is a LoRa library that allows you to load variables into the LoRa packet buffer in a direct manner, at the transmitter you do this;

  LoRa.startWriteSXBuffer(0);
  LoRa.writeBufferChar(trackerID, sizeof(trackerID));
  LoRa.writeFloat(latitude);
  LoRa.writeFloat(longitude);
  LoRa.writeUint16(altitude);
  LoRa.writeUint8(satellites);
  LoRa.writeUint16(voltage);
  LoRa.writeInt8(temperature); 
  TXPacketL = LoRa.endWriteSXBuffer();

And at the receiver you read out the variables from the received packet in the same order;

  LoRa.startReadSXBuffer(0);               //start buffer read at location 0
  LoRa.readBufferChar(receivebuffer);      //read in the character buffer
  latitude = LoRa.readFloat();             //read in the latitude
  longitude = LoRa.readFloat();            //read in the longitude
  altitude = LoRa.readUint16();            //read in the altitude
  satellites = LoRa.readUint8();           //read in the number of satellites
  voltage = LoRa.readUint16();             //read in the voltage
  temperature = LoRa.readInt8();           //read in the temperature
  RXPacketL = LoRa.endReadSXBuffer();      //finish packet read, get received packet length

But how would you deal with multiple measurements of different number of values, especially different kind of values?

E. G Sime node measure temperture and humidity, other node measure pressure, wind speed and current and another node measure temperature, humidity, voltage and light?

Or if the node doesn't measure specific value, should I keep the byte order in the packet and the unmeasured values set E.G. zero?

This example looks pretty good but I'm using the official Heltec library.

I will appreciate something similar using this comlatible library: Heltec_ESP32/src/radio/radio.c at master · HelTecAutomation/Heltec_ESP32 · GitHub

Thanks

Which library are you refering to ?

The library with the direct style of writing to the LoRa packet buffer is this one;

The Library has the same example shown above for SX127X, SX126X and SX128X.

There might not be individual examples for each of the many many different types of Arduino compatible boards, but it should not be a problem to use the sketches on an Arduino compatible.

I'm using this library: Heltec_ESP32/src/radio/radio.c at master · HelTecAutomation/Heltec_ESP32 · GitHub

So I don't want to change the functions for LoRa transmitting and receive because the communication works well. So I thought write similar part as your posted code using this library

based on their sender and receiver examples, you could try something like this

SENDER

// SENDER

#include "LoRaWan_APP.h"

#define RF_FREQUENCY                                915000000 // Hz
#define TX_OUTPUT_POWER                             5        // dBm
#define LORA_BANDWIDTH                              0         // 0: 125 kHz,1: 250 kHz,2: 500 kHz, 3: Reserved
#define LORA_SPREADING_FACTOR                       7         // SF7..SF12
#define LORA_CODINGRATE                             1         // 1: 4/5, 2: 4/6, 3:  4/7, 4: 4/8
#define LORA_PREAMBLE_LENGTH                        8         // Same for Tx and Rx
#define LORA_SYMBOL_TIMEOUT                         0         // Symbols
#define LORA_FIX_LENGTH_PAYLOAD_ON                  false
#define LORA_IQ_INVERSION_ON                        false

struct Paylaod {
  float temperature;
  float humidity;
};

bool lora_idle = true;

static RadioEvents_t RadioEvents;

void OnTxDone() {
  Serial.println("TX done......");
  lora_idle = true;
}

void OnTxTimeout() {
  Radio.Sleep( );
  Serial.println("TX Timeout......");
  lora_idle = true;
}

void setup() {
  Serial.begin(115200);
  Mcu.begin(HELTEC_BOARD, SLOW_CLK_TPYE);

  RadioEvents.TxDone = OnTxDone;
  RadioEvents.TxTimeout = OnTxTimeout;

  Radio.Init( &RadioEvents );
  Radio.SetChannel( RF_FREQUENCY );
  Radio.SetTxConfig( MODEM_LORA, TX_OUTPUT_POWER, 0, LORA_BANDWIDTH,
                     LORA_SPREADING_FACTOR, LORA_CODINGRATE,
                     LORA_PREAMBLE_LENGTH, LORA_FIX_LENGTH_PAYLOAD_ON,
                     true, 0, 0, LORA_IQ_INVERSION_ON, 3000 );
}

void loop() {
  if (lora_idle)  {
    delay(1000);
    Paylaod payload;
    payload.temperature = random(-1000, 1000) / 10.0;
    payload.humidity = random(0, 101) / 100.0;
    Radio.Send( (uint8_t *)&payload, sizeof payload);
    Serial.print("sending packet: temperature = ");
    Serial.print(payload.temperature, 2);
    Serial.print(" and humidity = ");
    Serial.println(payload.humidity, 2);
    lora_idle = false;
  }
  Radio.IrqProcess( );
}

RECEIVER

// RECEIVER

#include "LoRaWan_APP.h"
#define RF_FREQUENCY                                915000000 // Hz
#define LORA_BANDWIDTH                              0         // 0: 125 kHz,1: 250 kHz,2: 500 kHz, 3: Reserved
#define LORA_SPREADING_FACTOR                       7         // SF7..SF12
#define LORA_CODINGRATE                             1         // 1: 4/5, 2: 4/6, 3:  4/7, 4: 4/8
#define LORA_PREAMBLE_LENGTH                        8         // Same for Tx and Rx
#define LORA_SYMBOL_TIMEOUT                         0         // Symbols
#define LORA_FIX_LENGTH_PAYLOAD_ON                  false
#define LORA_IQ_INVERSION_ON                        false

struct Paylaod {
  float temperature;
  float humidity;
};

static RadioEvents_t RadioEvents;
bool lora_idle = true;


void OnRxDone( uint8_t *byteBuffer, uint16_t size, int16_t rssi, int8_t snr ) {
  Paylaod payload;
  if (size == sizof paylaod) {
    memcpy(&payload, byteBuffer, size );
    Serial.print("received packet: temperature = ");
    Serial.print(payload.temperature, 2);
    Serial.print(" and humidity = ");
    Serial.println(payload.humidity, 2);    
  } else {
    Serial.println("Wrong paylaod size");
  }
  Radio.Sleep( );
  lora_idle = true;
}

void setup() {
  Serial.begin(115200);
  Mcu.begin(HELTEC_BOARD, SLOW_CLK_TPYE);

  RadioEvents.RxDone = OnRxDone;
  Radio.Init( &RadioEvents );
  Radio.SetChannel( RF_FREQUENCY );
  Radio.SetRxConfig( MODEM_LORA, LORA_BANDWIDTH, LORA_SPREADING_FACTOR,
                     LORA_CODINGRATE, 0, LORA_PREAMBLE_LENGTH,
                     LORA_SYMBOL_TIMEOUT, LORA_FIX_LENGTH_PAYLOAD_ON,
                     0, true, 0, 0, LORA_IQ_INVERSION_ON, true );
}

void loop() {
  if (lora_idle) {
    lora_idle = false;
    Serial.println("into RX mode");
    Radio.Rx(0);
  }
  Radio.IrqProcess();
}

I typed this here so I don't know if it compiles, mind typos.

The idea is to define a structure and send that structure over. as it's the same platform on both side, I assume the structure will be organised in the same way - you could add GCC attributes (packed, align) to the structure to force the layout in a more stringent way.

➜ give it a try

Glamorous, it works nicely! I like the structure Payload{}.

Can you please explain me the difference between sending the values in char like:

#define BUFFER_SIZE                                 30 // Define the payload size here

char txpacket[BUFFER_SIZE];

sprintf(txpacket,"21.5676.24");  //start a package
   
Serial.printf("\r\nsending packet \"%s\" , length %d\r\n",txpacket, strlen(txpacket));

Radio.Send( (uint8_t *)txpacket, strlen(txpacket) ); //send the package out	

On the receivers side:

void OnRxDone( uint8_t *payload, uint16_t size, int16_t rssi, int8_t snr )
{
    rssi=rssi;
    rxSize=size;
    memcpy(rxpacket, payload, size );
    rxpacket[size]='\0';
    Radio.Sleep( );
    Serial.printf("\r\nreceived packet \"%s\" with rssi %d , length %d\r\n",rxpacket,rssi,rxSize);
    lora_idle = true;
}

I get this: received packet "21.5676.24" with rssi -24 , length 10
So here the length corresponds to the number of characters of received packet.

and your example, where the values are floats where I get:
received packet: temperature = -66.60 and humidity = 0.66
size=8

Here the size corresponds to the number of transmitted bytes.

Will I spare the the length of the sent string if I send data in float format instead of chars? Or how many bytes are occupied by one character sent in char format?

Thank you

this line

puts the ASCII code corresponding to the text "21.5676.24" and a trailing null char into the txpacket buffer

so the txpacket is really what we call a cString.

when you call

you ask the radio to send out a number of bytes from that buffer, that number being calculated by strlen(txpacket) which is the length of the text without the trailing null char. so you send basically the ASCII code for 21.5676.24


my code sends the bytes of a structure and so the number of bytes sent is the size of the structure

Here: Working with Bytes | The Things Network they say that it would take 3 bytes per character when sending text.

So my question is if will I spare the transmitted size of data by LoRa when sending values in float format (your code example) instead of chars (example I posted in previous post)?

I would also like to check it by some fucntion like sizeof() or length() but I'm confused in it.

Thanks