Storing things in a hex string

Using Arduino Programming Questions
1

I've got a LoRa-based project, where a small device collects some values (temperature, humidity, and so forth), and then pushes it over LoRa to a specific endpoint. Due to some constraints, the payload can only be a single hex string.

So far, I've defined the hex string as:

uint8_t payload[16];

Which results in a hex string of 10 characters. I got some example code, which tells me to do the following:

  payload[0] = type, HEX;
  payload[1] = lightLevel, HEX;
  payload[2] = temperature, HEX;
  payload[3] = humidity, HEX;

But I think this'll result in only storing a single digit. Is there a way to store the full value in pre-determined spots?

For example:

type = 1
Lightlevel = 912
temperature = 23.8
humidity = 55

The resulting payload would then be something like:

1f00912f0238f055

with the f separating the different value types. Is this doable? If so, how do I concat these strings? Is there another, easier way?

2

'f' is a valid hex digit.
You can't use it as a separator in a hex string.

Have you got some prejudice against commas?

3 
payload[0] = type, HEX;
  payload[1] = lightLevel, HEX;
  payload[2] = temperature, HEX;
  payload[3] = humidity, HEX;

If you found example code that told you to do this then you should avoid any other examples by that author. This just stores the number 16 in each of those variables. If you want to convert a number to a hex string in ascii then use itoa or sprintf.

To assemble it all together in one string with separators use sprintf.

4

Use a struct and send all the sensor data in binary.

5

Your current code will not compile, you should provide a full listing that gives a clean compile.

As @tmfkaawol said you cannot use 'f' as a separator.
If you cannot use a comma then consider using fixed width fields.
Start by deciding what type each variable is and then the range of possible values it can take.
You can then either represent the binary value of the variable as hexadecimal or you can consider the variable as a string of characters and use the hex value of each of the ASCII characters in the string.

6

So far, I've defined the hex string as:

You have defined a character or byte array capable of storing 16 bytes. If you are defining a fixed format payload to be transmitted to another device, there is no need for separators. Just choose the correct size for each of the variables and be consistent between transmitter and receiver. Example: 1 byte for light level, two bytes for temperature, one byte for humidity.

This just stores the number 16 in each of those variables

I thought that too (as a result of the comma operator), but one of the forum C++ pros corrected me.

The "=" has higher precedence, so the ",HEX" does nothing unless parentheses dictate otherwise. Agreed, though, avoid anything else by the author of that code.

Try this:

void setup() {
  Serial.begin(9600);
  int x;
  x = 100, HEX;
  Serial.print(" x = "); Serial.println(x);
  x = (101, HEX);
  Serial.print(" x = "); Serial.println(x);
}

void loop() {
}
7

Well, I'm comforted by the fact that I'm not the only one reading the example code with a raised eyebrow!

The payload format can be fixed by me, and I can even set the length of the payload (with some oddly chosen potential lengths, but still). So then I can use fixed width for the different elements of the payload (and, apologies for the f; I realized this would not provide any form of flexibility, would likely backfire, and wasn't really needed anyway).

Right now I'm storing the values as uint, but it's the combining that's got me puzzled. The (somehow) working example code combines values in an uint8_t array, and I've established the supportive library does not approve of using chars.

So, my question works as follows:
I have a set of uint values, of pre-determined length. How do I combine them as a single uint8_t array with pre-defined field widths?

The only thing I can come up with is a large and unflexible horror that adds hex digits one by one to an empty array, but then I'm stuck at how to get a single digit. Also, I'm hoping something more elegant is possible.

8

supersecretsecretforgot:
Also, I'm hoping something more elegant is possible.

One more try:

gfvalvo:
Use a struct and send all the sensor data in binary.

9

supersecretsecretforgot:
Right now I'm storing the values as uint,

...apart from that temperature value with decimals.

10

Forget all about using "hex digits", they are for humans to read.

11

TheMemberFormerlyKnownAsAWOL:
...apart from that temperature value with decimals.

I just multiply by a hundred, and the values come through okay.

I'm not entirely clear on how a struct works, and I've neglected reading up on it (which I'm fixing now). It feels like the Matlab struct, which is this magical catch-all for whatever data structure you've got going on, but I suspect that implementation generates quite a bit of overhead.

Also, I'm seeing an example of pushing a struct over a serial bus, but unfortunately I'm working with a library that only accepts a hex array (and I'm not allowed, or likely capable enough, to poke at it).

How does one make an union of a struct?

(byte*)&payloadStruct, sizeof(payloadStruct))

or do I do something with typedef union?

12

Hopefully you are aware that temperatures can exceed 25.5 degrees as well as go negative?

13

supersecretsecretforgot:
but unfortunately I'm working with a library that only accepts a hex array (and I'm not allowed, or likely capable enough, to poke at it).

Perhaps you'd be so kind as to share your full code and this library?

14

gfvalvo:
Perhaps you'd be so kind as to share your full code and this library?

The library is not supposed to be shared at the moment, regrettably. I'm tempted to read your response as "can't be done, change the library is the only option.". Is that the case?

TheMemberFormerlyKnownAsAWOL:
Hopefully you are aware that temperatures can exceed 25.5 degrees as well as go negative?

Depending on the range of temperature variation, this may not be an issue. And aside from multiplying by a hundred, one can also multiply by other values, and even subtract, if that transformation would fit the value of whatever you're interested in to the range of possible values allowed.

15

supersecretsecretforgot:
The library is not supposed to be shared at the moment, regrettably.

Then I suggest you pay someone to fix your problem after signing an NDA.

16

unfortunately I'm working with a library that only accepts a hex array

It is very likely that you misunderstand how this library works, because that statement makes no sense.

All data are binary.

17

I asked for some help in working with a closed-off library that's under NDA, and I got:

jremington:
It is very likely that you misunderstand how this library works, because that statement makes no sense.

gfvalvo:
Then I suggest you pay someone to fix your problem after signing an NDA.

Which is not helpful to the current problem. Due to NDAs and such I can't push back against these statements, which is uncomfortable.

For anyone who's interested, I've temporarily solved the issue by putting the characters one by one in a hex array, and padded them with zeroes to conform to the fixed width fields, like so:

185415561500087756

Which corresponds to a message type of 1, a temperature of 8.541 degrees, a humidity of 5.615, and a light level of 87756 Lux. I'm not using the full resolution possible with hex values, and I'm looking into structs to see if I can do that at a later moment, but at least this works, and has the benefit of the resulting string (which appears on the other side of the system) being human-readable.

Anyway, thanks for your help.

18

Maybe it would help to use concrete examples.
Let’s say your temperature can go from 0.0 – 25.5 degrees, and you only need it accurate to one decimal place.
You could multiply the temperature by 10 which would give you an integer in the range 0-255.
This range could be stored in an unsigned byte.

So if the actual reading is 19.3 degrees you could store it in a byte with the value 193 (base 10).
You now have a couple of choices on how to send the reading as a hex string.
You could send it as two ASCII characters C1 or three ASCII characters 193, the receiving end needs to which you are going to be doing and that the reading has been multiplied by 10, it can then reconstruct the floating point number 19.3. Because the temperature reading will always take the same number of characters to transmit you don’t need a separator between it and the next reading.
A structure will contain several variables of different types for your readings but the length of each type is fixed so it is not much different to sending a single byte. You do though need to consider what would be an appropriate type for each variable given the range of the reading.

19

supersecretsecretforgot:
For anyone who's interested, I've temporarily solved the issue by putting the characters one by one in a hex array...

Again, there's no such thing as a "hex array".

20

gfvalvo:
Again, there's no such thing as a "hex array".

I'm storing values in an a variable which is defined like this:

uint8_t payload[payloadLength]

Yes, this is an array of unsigned 8-bit integers. I'm not exactly treating it as an array of integers, but I'll call it whatever you want as long as it means the discussion you seem to be fond of having gets bypassed.

ardly:
You could send it as two ASCII characters C1 or three ASCII characters 193, the receiving end needs to which you are going to be doing and that the reading has been multiplied by 10, it can then reconstruct the floating point number 19.3. Because the temperature reading will always take the same number of characters to transmit you don’t need a separator between it and the next reading.

Sending three ascii characters was the previous implementation, now I'm sending two and having the other end reconstruct the original measurement. I've only suggested the use separators initially, and have since realized these are not needed.

ardly:
A structure will contain several variables of different types for your readings but the length of each type is fixed so it is not much different to sending a single byte.

I'm reading up on the struct and how it works, and it seems like a good candidate for the next implementation, as the code would be quite a bit more concise and elegant.