Converting uint8_t array to char array - Problems

Hello,
I use those 2 librarys : BOSCH BSEC (for BME680) and iotWebconf (which manages Wifi and settings)

to save the calibration of the bosch bsec library (which is a uint8_t [139] ) i need to write it to a char, since iotwebconf works with chars to save settings

But i cant manage to get it working even with alot googling and with approaches that "should" work but dont!

i managed to get a output of the uint8_T in serial as hex with following function (which is the only thing that worked so far..)

for(int i; i<=sizeof(bsecState1_uint8_t); i++){
   Serial.print("0x");
   Serial.println(bsecState1_uint8_t[i], HEX);
}

the output looks like this:

0x0
0x8
0x4
0x1
0x3D
0x0
0x0
0x0
0x0
0x0
0x0
0x0
0x73
0x0
0x0
0x0
0x2D
0x0
0x1
0x1
0x29
0xA8
0xB9
0x40
0x8D
0xA8
0xB9
0x40
0x56
0xEC
0xB4
0x40
0xD1
0xEA
0xB4
0x40
0x0
0x0
0x0
0x0
0x2C
0x1
0x15
0x15
0x0
0x0
0x40
0x40
0x0
0x0
0x40
0x40
0x0
0x0
0x3
0x0
0x17
0xA8
0x5F
0x3F
0x14
0xC
0x0
0x2
0x1
0x37
0x19
0x12
0x42
0xF3
0x2E
0x7
0x42
0x10
0x0
0x3
0x1
0xD6
0x37
0x9E
0x40
0x9F
0xE8
0xC7
0x41
0x3D
0xCF
0x6C
0x42
0x16
0x0
0x5
0x1
0x80
0x4D
0x81
0x83
0x46
0x0
0x0
0x0
0x0
0x0
0x0
0x0
0x58
0xE1
0x8B
0x52
0x1
0x1
0xC
0x0
0x9
0x1
0x80
0x4D
0x81
0x83
0x46
0x0
0x0
0x0
0x8
0x0
0xA
0x1
0x4D
0xDC
0xC7
0x41
0x0
0x0
0x0
0x0
0x78
0x34
0x0
0x0
0x0

but something like the following wont work but my googling says me it "should" work (i just get a empty serial line)

    String str1 = (char*)bsecState1_uint8_t;
    Serial.println(str1);

(i know thats a string not a char, but even that i dont manage to get working for testing the outputs in serial, and i think there lays the mainproblem)

this here wont work either:

char bsecState2_test[300];
memcpy(bsecState1_test, bsecState1_uint8_t, sizeof(bsecState1_uint8_t));

So is there something im missing here? the output is there since the hex function works but nothing else

Thanks in advance

I guess the question is; what are you trying to do? It sounds like you are trying to get the BME680 factory calibration data off the Arduino to some other computer. If this is the case what is the ultimate goal?

there is a function itostr but it turns your data into a string so the number 54864 will be 5 char's.

And from what I understand C++ is not very good at handling strings so working with an array of strings is problematic.

I just try to convert the int8_t to a char ( so iotwebconf is able to save it ) on the same ESP32 nothing more (and converting it back to uint8_t if the program is started so bsec can use the calibration)

but i got confused since even conversions to string dont work (has this something todo with the first result of the hex function? (since it is 0x0, or is null terminated a different value?)

from what i understood memcpy would be the best since it just mirrors the needed value to a char, which should work since uint8_t and char is basicly the same, atleast i understood it that way

BTW:

    uint8_t bsecState2_uint8_t[8] = {'0','0','0','0','7','0','1','5'};
    Serial.println(String((char*)bsecState2_uint8_t));

IS actually working, but it doesnt work if i use the uint8_t by the bsec library :frowning:

Can you provide links to those two libraries you use?

Sure!

Before we go any further, can you post a few of the numbers you are trying to convert? I don't recall what the BME680 used but I remember the BME280 used some 5 digit decimal numbers for the temp and humidity calibration.

But your question states the calibration numbers are uint8_t meaning they are between 0 and 255 (dec) or 0 to 0xFF. So what can iotwebconf store?

Question:
Since the calibration factors are only good for a specific part and the calibration factors are stored on said specific part, what is the benefit you are getting from storing them on and external platform?

you missunderstood something, iotWebConf just manages settings for you to store it in EEPROM (it does it because you can change settings on the webpage too)

You can also create so called "hidden parameters" which are hidden from the webpage

since i would need to mess with the library files if i wanna save it myself in eeprom (so my eeprom calls and the ones from iotwebconf dont collide) i would rather use the function of iotwebconf to store those "hidden parameters" for that i need to convert the uint8_t from bsec to char, since iotwebconf handles the parameters as chars, its all on the same ESP32 i dont extract the calibration values to anywhere else

I just wanna store them since a power off would need a "long" calibration proccess again, if you store them they can be used right away again

basicly i just need to convert the hex output from above (which is the calibration array), i have two sensors so i need todo it twice, for each sensor but thats it, no other values

and i think iotWebConf has no problem with storing those values ( since it also stores them as char arrays)

the main problem here is that i dont get any output from my conversion trys, if i would get a output (even just a serial output of the calibration number as string/char beside the hex function above would be great) i would be one step closer to save them :slight_smile:

and the hex function in the startpost just seems to work since it reads byte for byte i guess

It's still not clear to me what you want to do.
You have a calibration array of 139 bytes.

If we look at some of the first numbers you presented
0,8,4,1,61,0,0,0,0,0,0,0,115...
In what format do they need to be saved?
Is 61 two chars '6' and '1' and 115 three chars '1','1','5'.
Are you trying to convert integers to text?

I want to save them in such a way that they can be reconstructed to the original uint8_t again, so the easiest would be to save byte per byte i guess, in the end i dont wanna change it at all, i just need the conversion to be able to save them (and convert them back when they are needed)

EDIT: and they also cant be change or it wouldnt work or give me wrong results if i try to load a "changed" bsec state, they need to be 100% like the original after conversion back

But now as you said it there lays the problem right? uint8_t and char would be just interchangeable if values go from 0-9 if the full 255 gets used for a byte i cant just convert it to char like i tried, right?

So i would need a char which is 3 times in size as the uint8_t and then convert 0-9 number for 0-9 number right?

It's still not clear to me what iotWebConf wants for the data format.
Are you trying to convert integers with values from 0 to 255 into to text representations of those numbers?

strncpy(
  iotWebConf.getThingNameParameter()->valueBuffer,
  "My changed name",
  iotWebConf.getThingNameParameter()->getLength());

thats a example of how to change values, thats also possible with custom created parameters

actually i guess iotWebConf wants a string i just assumed i can also save it as char since you have to declare custom parameters with char and an array size, but maybe iotWebConf just works with strings and makes the conversionen on its own

(is this bad? since i readed that strings should be avoided)

So if we can get those number into a string it would be fine (for now i guess) the problem with string is if a "0x00" is there it would stop the conversionen ( thats why i get nothing on my trys, since the first value IS 0x00 (i found a google answer here))

The sprintf() function can be used to convert the elements of a byte array to a NULL terminated char array.

atoi() can be used to convert the null terminated char arrays to the byte values.

If you look here, it says that

The char datatype is a signed type, meaning that it encodes numbers from -128 to 127.

On the other hand , uint8_t is unsigned. Could that be the problem?

So you are trying to print some data as a string and it doesn’t work. Why are you saying google told you it should work is unknown. You should read about strings and understand that strings are a subclass of char array with own rules that while you can convert any byte data to a string doesn’t mean it would be printable

Would be much easier to just store this in EEPROM in the uint8_t form, but if you insist on having it as char:
You could easily take each byte and convert each nibble (4 bits) into the corresponding ASCII character to represent its hex value, but since the date has no need to be human readable there is no need to go through the complication of actually using 0 - 9 and A - F, just use a contiguous 16 ASCII characters. In this code I'm using 0123456789 ; : < = > ? (the six ASCII character after the numbers).
Reversing the process would be similar.

uint8_t bsecState1_uint8_t[139];

void setup() {
  Serial.begin(9600);
  //load array with random numbers
  for (size_t i = 0; i < sizeof(bsecState1_uint8_t); i++) {
    bsecState1_uint8_t[i] = random(256);
  }
  
  char bsecState1_char[sizeof(bsecState1_uint8_t) * 2 + 1];
  
  for (size_t i = 0; i < sizeof(bsecState1_uint8_t); i++) {
    //convert each nibble to a char from '0' through '?'
    bsecState1_char[i * 2] = (bsecState1_uint8_t[i] >> 4) + '0';
    bsecState1_char[i * 2 + 1] = (bsecState1_uint8_t[i] & 0x0F)  + '0';
    Serial.print(i);
    Serial.print('\t');
    Serial.print(bsecState1_uint8_t[i], HEX);
    Serial.print('\t');
    Serial.print(bsecState1_char[i * 2]);
    Serial.println(bsecState1_char[i * 2 + 1]);
  }
  bsecState1_char[sizeof(bsecState1_uint8_t) * 2] = '\0'; //terminate string
  
  Serial.println(bsecState1_char);
  Serial.println(strlen(bsecState1_char));
}

void loop() {
}

Here's some simple code which converts an array of uint8_t integers into an array of null terminated hex character strings using sprintf() and converts them back again to the integers with strtol().

It's not as memory efficient as David's example, but it is human readable, and can be accessed by individual value incase you don't need the complete calibration or just want to change a few.

void setup() {
  Serial.begin(115200);
  byte startBytes[] = {0, 8, 4, 1, 61, 0, 0, 0, 0, 0, 0, 0, 115, 0, 0, 0, 45, 0, 1 , 1, 41, 168, 185, 100, 141};

  char endChars[sizeof(startBytes)][3] = {0};//2 hex digits plus null

  for (byte i = 0; i < sizeof(startBytes); i++)
  {
    sprintf(endChars[i], "%x", startBytes[i]);//hex string
  }

  for (byte i = 0; i < sizeof(startBytes); i++)
  {
    Serial.print(i);
    Serial.print('\t');
    Serial.print(startBytes[i]);
    Serial.print('\t');
    Serial.print(endChars[i]);
    Serial.print('\t');
    Serial.println(strtol(endChars[i], NULL, 16)); //conver hex strng to int

  }

}

void loop() {
  // put your main code here, to run repeatedly:

}

Wait a sec...

Wouldnt this be sufficent?

char bsecState2_test[200];
  for (byte i = 0; i < sizeof(bsecState2_uint8_t); i++)
  {
      bsecState2_test[i] = (char)bsecState2_uint8_t[i];
  }
  Serial.println(" bsecstate2_test ");
  Serial.println(bsecState2_test);

I mean, char stores each character as byte and uint8_t stores each number as byte, so a conversion isnt needed, just store each number (even if its 255) as the "char" representation in the same byte, or do i think too simple here?

(im testing it, but i have to wait for the calibration to trigger, which takes some hours in worse case :/)

Edit: Well the zeros seem to be the only problem, unless there is no zero this would work i think :confused: