Arduino Uno with TheThingsNetwork memory overwrite issue

Hi, I am struggling with C++ memory allocation issue. Below is a test program demonstrating the issue. It makes use of TheThingsNetwork library. I am trying to send the robotId via the things network. The sketch demonstrates what I am trying to achieve. In this basic test case it converts a Hex string to a uint8_t array and then sends it out.

This sketch does not work and results in output, where 07070717 is not the correct robot id:
11:26:38.256 → Sending: mac tx uncnf 1 07070717
11:26:40.391 → Successful transmission

#include <TheThingsNetwork.h>

// Debug
#define debugSerial Serial

// Things network definitions
const char *robotId = "00000001";
const char *appEui = "XXXXXXXXXXXXXXXXXXXXXXXXXXXX";
const char *appKey = "XXXXXXXXXXXXXXXXXXXXXXXXXXXX";
#define loraSerial Serial1
#define freqPlan TTN_FP_EU868
TheThingsNetwork ttn(loraSerial, debugSerial, freqPlan);


void setup() {
    loraSerial.begin(57600);
    debugSerial.begin(9600);

    // Wait a maximum of 10s for Serial Monitor
    while (!debugSerial && millis() < 10000);
    
    debugSerial.println("-- STATUS");
    ttn.showStatus();

    debugSerial.println("-- JOIN");
    ttn.join(appEui, appKey);
    delay(10000);
}

void hexToUint8(const char * in, uint8_t * out){
    for(uint8_t i = 0; i < strlen(in)/2; i++){
        char tmp[3];
        tmp[0] = in[i*2];
        tmp[1] = in[i*2+1];
        tmp[3] = 0;
        out[i] = strtol(tmp,NULL,16);
    }
}

void loop() {
    
    int length = (int)strlen(robotId)/2;
    uint8_t * out = new uint8_t[length];
    hexToUint8(robotId, out);
    ttn.sendBytes(out, length );
    delete[] out;
    
    delay(5000);

}

When changing the loop portion as shown below it does work and I cant get my head around what is happening. It seems like the memory gets overwritten. I suspect that declaring the test variable allocates the memory but I am not a C++ programmer.

void loop() {
  
    char test2[9] = "ABCDEF01";
    
    int length = (int)strlen(test2)/2;
    uint8_t * out = new uint8_t[length];
    hexToUint8(robotId, out);
    ttn.sendBytes(out, length );
    delete[] out;

    strlen(test2);
    
    delay(5000);

}

this results in output which includes the correct robot id:
11:25:11.537 → Successful transmission
11:25:16.565 → Sending: mac tx uncnf 1 00000001

I hope you guys can shed some clarity in what is happening here, and how I can makes this work without the need for the test variable?

I can't see how you're getting 07070717, but your hex to int routine is flawed. Try printing the content of out. You'll get nothing because that function takes the first two zeros and puts a character zero in the string which terminates it. Try:

    out[i] = strtol(tmp, NULL, 16)+48;

You will then need to zero terminate out.

wildbill:
I can’t see how you’re getting 07070717, but your hex to int routine is flawed. Try printing the content of out. You’ll get nothing because that function takes the first two zeros and puts a character zero in the string which terminates it. Try:

    out[i] = strtol(tmp, NULL, 16)+48;

You will then need to zero terminate out.

Thank you very much, I do have a problem with my conversion. I must not understand or do something wrong with the "+48"trick it yields totally wrong output (which is logical). The closest to a solution I get is with this test sketch. The problem occurs, like you said, when the input equals ‘00’, which terminates the array. I have no idea how to fix this. I understand by adding 48, you effectively prevent the array from being terminated but then the other values are wrong as well. How can I fix this?

I do have a backend server running behind TheThingsNetwork that would be able to subtract 48 from every value, but this seems like untidy work. Is this the only way to go?

#define debugSerial Serial

void setup() {
    debugSerial.begin(9600);
}

void hexToUint8(const char * in, uint8_t * out){
  
    serialOut(in);

    for(uint8_t i = 0; i < strlen(in)/2; i++){
        char tmp[3];
        tmp[0] = in[i*2];
        tmp[1] = in[i*2+1];
        out[i] = strtol(tmp,NULL,16);
    }

    serialOut(out);
}


void test(const char * input, int index){
    debugSerial.println("");
    debugSerial.print("TEST ");
    debugSerial.print(index);
    debugSerial.println("");
    
    int length = (int)strlen(input)/2;
    uint8_t * out = new uint8_t[length];
    hexToUint8(input, out);
    delete[] out;
}

void loop() {

    test("FF", 1);
    test("FF11", 2);
    test("FF11FF",3);
    test("FF01FF",4);
    test("FF10FF", 5);
    test("00FF00",6);
    delay(5000);

}

// Output to serial
void serialOut(const char * out){
    debugSerial.print("   CHARS: ");
    int index = 0;
    while(out[index] != 0){
      //debugSerial.print(out[index], HEX);
      debugSerial.print(out[index]);
      debugSerial.print(" ");
      index++;
    }
    debugSerial.println("");
}


void serialOut(uint8_t * out){
    debugSerial.print("   UINT: ");
    int index = 0;
    while(out[index] != 0){
      //debugSerial.print(out[index], HEX);
      debugSerial.print(out[index]);
      debugSerial.print(" ");
      index++;
    }
    debugSerial.println("");
}

This yields the following output. Problem is visible with test 6:

18:28:59.924 -> TEST 1
18:28:59.924 ->    CHARS: F F 
18:28:59.924 ->    UINT: 255 
18:28:59.924 -> 
18:28:59.924 -> TEST 2
18:28:59.924 ->    CHARS: F F 1 1 
18:28:59.924 ->    UINT: 255 17 
18:28:59.924 -> 
18:28:59.924 -> TEST 3
18:28:59.924 ->    CHARS: F F 1 1 F F 
18:28:59.924 ->    UINT: 255 17 255 
18:28:59.924 -> 
18:28:59.924 -> TEST 4
18:28:59.924 ->    CHARS: F F 0 1 F F 
18:28:59.924 ->    UINT: 255 1 255 
18:28:59.924 -> 
18:28:59.924 -> TEST 5
18:28:59.924 ->    CHARS: F F 1 0 F F 
18:28:59.924 ->    UINT: 255 16 255 
18:28:59.924 -> 
18:28:59.924 -> TEST 6
18:28:59.924 ->    CHARS: 0 0 F F 0 0 
18:28:59.924 ->    UINT:

The problem is in your original conversion function:

void hexToUint8(const char * in, uint8_t * out) {
  for (uint8_t i = 0; i < strlen(in) / 2; i++) {
    char tmp[3];
    tmp[0] = in[i * 2];
    tmp[1] = in[i * 2 + 1];
    tmp[3] = 0;
    out[i] = strtol(tmp, NULL, 16);
  }
}

Change:

    tmp[3] = 0;

to:

    tmp[2] = 0;

ToddL1962:
The problem is in your original conversion function:

void hexToUint8(const char * in, uint8_t * out) {

for (uint8_t i = 0; i < strlen(in) / 2; i++) {
   char tmp[3];
   tmp[0] = in[i * 2];
   tmp[1] = in[i * 2 + 1];
   tmp[3] = 0;
   out[i] = strtol(tmp, NULL, 16);
 }
}




Change:


tmp[3] = 0;




to:



tmp[2] = 0;

You are my personal hero. What a stupid mistake, thank you, this fixed it!