Resizing arrays

This post is not exactly for the Atmel hardware but for an ARM MCU from TI and is being programmed using the Energia IDE. However Energia is based on the Arduino IDE and has the exact same interface and syntax. If it works in Arduino, it'll work in Energia (well, almost). Which is why I'm hoping someone can help me here.

I'm setting up a webserver through which I'm collecting an user supplied SSID and password. This POST response (which looks like this - ssid=XYZ&pswd=ABC) is stored in a buffer which is then split into two separate arrays (acqSSID and acqPassword). However, when you run the code to print out the number of elements it shows 256 (presumably because the char array was defined with that many number of elements) though the Serial.println prints out just the SSID. Is there a way I can resize this array to just the size of the SSID characters in it? If I do not size the array at declaration I get garbage values.

The processing to split the POST array starts at line 327

The code file is attached here.

SimpleWebServerv3.ino (14.6 KB)

If it works in Arduino, it'll work in Energia

Surely you mean If it works in C++, it'll work in Energia

If I do not size the array at declaration I get garbage values.

Where and how do you see them ?

char acqSSID2[] = {0};

A one element array. How useful is that?

         //erase all excess characters from the buffer.
          for (int i=bufferlen ; i<MAX_STORE_BUFFER_LEN ; i++)
          {
            buffer[i] = 0;
          }

This indicates that you clearly do not understand what NULL terminated char array means. ONE null is enough.

You are pissing away SRAM unnecessarily. ALL string literals used in client.println() should be wrapped in the F() macro, to keep them out of SRAM.

We need to see your serial output to make any sense of your code and your post.

UKHeliBob:
Surely you mean If it works in C++, it'll work in Energia

Yes :slight_smile:

PaulS:

char acqSSID2[] = {0};

A one element array. How useful is that?

This was just for temporary testing but I've since removed it.

PaulS:

         //erase all excess characters from the buffer.

for (int i=bufferlen ; i<MAX_STORE_BUFFER_LEN ; i++)
         {
           buffer[i] = 0;
         }




This indicates that you clearly do not understand what NULL terminated char array means. ONE null is enough.

You are pissing away SRAM unnecessarily. ALL string literals used in client.println() should be wrapped in the F() macro, to keep them out of SRAM.

We need to see your serial output to make any sense of your code and your post.

Strangely when I was trying to terminate this with just one null it wasn't working but today it seems to be working.

Here is the Serial output:

Attempting to connect to WiFi network....
Connected to wifi
Waiting for IP...
My IP is: 192.168.1.107
Webserver started on http://192.168.1.107 and port: 80


Buffer length: 47
Stored POST buffer (storebuffer): ssid=ThisIsMyWiFiSSID&pswd=ThisIsMyWiFiPassword

acqSSID: ThisIsMyWiFiSSID
acqPassword: ThisIsMyWiFiPassword
Number of elements in acqSSID: 256

And the modified code:

    if(bufferlen != 0)
    {
      Serial.print("Buffer length: ");
      Serial.println(bufferlen);
      
      Serial.print("Stored POST buffer (storebuffer): ");
      Serial.write(storebuffer);
      Serial.println("\n");

      //copy everything after 'ssid=' till you reach '&pswd='. 5 is the offset which is the number of characters in 'ssid='. 
      //6 because its the next character being read from the storebuffer.
      int ctr = 0;
      for (ctr = 0; ctr < bufferlen; ctr++)
      {
        acqSSID[ctr] = storebuffer[ctr+5];
        //see if we've reached '&pswd' in the storebuffer. If yes, then break out of the loop since we've now acquired the SSID in the new array.
        if ( (storebuffer[ctr+6] == '&') && (storebuffer[ctr+7] == 'p') && (storebuffer[ctr+8] == 's') && (storebuffer[ctr+9] == 'w') && (storebuffer[ctr+10] == 'd') && (storebuffer[ctr+11] == '=') )
        {
          break;
        }
      }
      
      //Terminate the array.
      acqSSID[ctr+1] = '\0';

      //copy everything after '&pswd' into acqPassword. The number 12 is the offset which is the addition of  
      //character count in 'ssid=' and '&pswd=' plus one increment from the loop where we acquired acqSSID 
      //since there the count increments and then the loop breaks.
      for (int i = 0; ctr < bufferlen; i++)
      {
        acqPassword[i] = storebuffer[ctr+12];
        ctr++;
      }

      //Terminate the array.
      acqPassword[bufferlen] = '\0';

      Serial.print("acqSSID: ");
      Serial.println(acqSSID);
      Serial.print("acqPassword: ");
      Serial.println(acqPassword);

      int len_a = sizeof(acqSSID) / sizeof(acqSSID[0]);
      Serial.print("Number of elements in acqSSID: ");
      Serial.println(len_a);

    }
      Serial.print("Stored POST buffer (storebuffer): ");
      Serial.write(storebuffer);
      Serial.println("\n");

When you print strings (or god-awful Strings) you should print starting and ending markers that are printable characters. \n is NOT a printable character.

      Serial.print("storebuffer: [");
      Serial.write(storebuffer);
      Serial.println("]");

We really don't care what you think the array contains. We care what array you are printing.

        //see if we've reached '&pswd' in the storebuffer. If yes, then break out of the loop since we've now acquired the SSID in the new array.
        if ( (storebuffer[ctr+6] == '&') && (storebuffer[ctr+7] == 'p') && (storebuffer[ctr+8] == 's') && (storebuffer[ctr+9] == 'w') && (storebuffer[ctr+10] == 'd') && (storebuffer[ctr+11] == '=') )

Dumb. Use memcmp()!

      //Terminate the array.
      acqSSID[ctr+1] = '\0';

No. Do this after adding each character to it!

      Serial.print("acqSSID: ");
      Serial.println(acqSSID);

The delimiters are missing.

      int len_a = sizeof(acqSSID) / sizeof(acqSSID[0]);
      Serial.print("Number of elements in acqSSID: ");
      Serial.println(len_a);

The 256 that is printed IS the number of elements in the array. strlen(acqSSID) would tell you how many were used by the string.

Why not use pointers?

As far as I understand your code, store_buffer eventually contains the credentials in the format ssid=XYZ&pswd=ABC. So I would rename it to credential_buffer. Next use pointer to point to the credential values.

#define CREDENTIALSIZE 64

// buffer with credentials
char credential_buffer[CREDENTIALSIZE] = "ssid=XYZ&pswd=ABC";

// pointers to the actual credential values in the buffer
char *accSSID;
char *accPWD;

void setup()
{
  Serial.begin(9600);

  Serial.print("Received credentials: "); Serial.println(credential_buffer);
  Serial.print("credential_buffer size: "); Serial.println(sizeof(credential_buffer));
  Serial.print("credential size: "); Serial.println(strlen(credential_buffer));

  // check if credentials contains 'ssid='
  accSSID = strstr(credential_buffer, "ssid=");
  if(accSSID  == NULL)
  {
    // something wrong
    Serial.println("'ssid=' not found");
    // bail out
    return;
  }
  // increment accSSID so it points to the data
  accSSID += strlen("ssid=");

  // check if credentials contains '&pswd='
  accPWD = strstr(credential_buffer, "&pswd=");
  if(accPWD == NULL)
  {
    // something wrong
    Serial.println("'&pswd=' not found");
    // bail out
    return;
  }

  // accPWD points to '&'; replace by NUL character so SSID value is null terminated
  *accPWD = '\0';

  // accPWD points to '&'; increment so it points to the data
  accPWD += strlen("&pswd=");

  // display result
  Serial.print("SSID: "); Serial.println(accSSID);
  Serial.print("PWD:  "); Serial.println(accPWD);
  
}

void loop()
{

}

Notes:
1)
the size of the credential_buffer is reduced to 64; adjust to your needs.
2)
the credential_buffer is 'corrupted'; if you print it after the processing, it will only print the part up to the SSID.
3)
The only real way to resize arrays is by dynamically allocating them (malloc, calloc, realloc, free). Stay away from it unless you absolutely know what you're doing.

PaulS:

        //see if we've reached '&pswd' in the storebuffer. If yes, then break out of the loop since we've now acquired the SSID in the new array.

if ( (storebuffer[ctr+6] == '&') && (storebuffer[ctr+7] == 'p') && (storebuffer[ctr+8] == 's') && (storebuffer[ctr+9] == 'w') && (storebuffer[ctr+10] == 'd') && (storebuffer[ctr+11] == '=') )



Dumb. Use memcmp()!

A little confused as to how I can do this. memcmp needs two arrays for comparison right?

PaulS:

      //Terminate the array.

acqSSID[ctr+1] = '\0';



No. Do this after adding each character to it!

Did not get what you meant. Right now its null terminating it after it exits the loop where it copies all the characters. The ctr value would be the position of the null terminating character in the array right?

sterretje:
Why not use pointers?

As far as I understand your code, store_buffer eventually contains the credentials in the format ssid=XYZ&pswd=ABC. So I would rename it to credential_buffer. Next use pointer to point to the credential values.

#define CREDENTIALSIZE 64

// buffer with credentials
char credential_buffer[CREDENTIALSIZE] = "ssid=XYZ&pswd=ABC";

// pointers to the actual credential values in the buffer
char *accSSID;
char *accPWD;

void setup()
{
  Serial.begin(9600);

Serial.print("Received credentials: "); Serial.println(credential_buffer);
  Serial.print("credential_buffer size: "); Serial.println(sizeof(credential_buffer));
  Serial.print("credential size: "); Serial.println(strlen(credential_buffer));

// check if credentials contains 'ssid='
  accSSID = strstr(credential_buffer, "ssid=");
  if(accSSID  == NULL)
  {
    // something wrong
    Serial.println("'ssid=' not found");
    // bail out
    return;
  }
  // increment accSSID so it points to the data
  accSSID += strlen("ssid=");

// check if credentials contains '&pswd='
  accPWD = strstr(credential_buffer, "&pswd=");
  if(accPWD == NULL)
  {
    // something wrong
    Serial.println("'&pswd=' not found");
    // bail out
    return;
  }

// accPWD points to '&'; replace by NUL character so SSID value is null terminated
  *accPWD = '\0';

// accPWD points to '&'; increment so it points to the data
  accPWD += strlen("&pswd=");

// display result
  Serial.print("SSID: "); Serial.println(accSSID);
  Serial.print("PWD:  "); Serial.println(accPWD);
 
}

void loop()
{

}




Notes:
1)
the size of the credential_buffer is reduced to 64; adjust to your needs.
2)
the credential_buffer is 'corrupted'; if you print it after the processing, it will only print the part up to the SSID.
3)
The only real way to resize arrays is by dynamically allocating them (malloc, calloc, realloc, free). Stay away from it unless you absolutely know what you're doing.

Thanks! This really does help. However, how do I copy characters from the storebuffer to a new array? Would something like this work? Also how would I copy the PSWD characters to another buffer since there is now no termination character to compare to?

  // accPWD points to '&'; replace by NUL character so SSID value is null terminated
  *accPWD = '\0';
  int i = 0;
  while (*accSSID != '\0')
  {
	SSIDbuffer[i] = *accSSID; //declared another buffer to store the SSID data.
	accSSID++;
	i++;
  }

What I ended up doing is converting the array to a String and then back to a char array with the length of the string. Not sure if this is an efficient way to do it but it seems to be working.

The new problem now is that when I enter characters such as #, $, @ etc. they get converted into HEX values. For e.g. when I entered '#' the array now contains %23. Any way I can get the exact value entered?

electrophile:
However, how do I copy characters from the storebuffer to a new array?

Why would you. There is a reason why I renamed it to credential_buffer. It stores the credentials and nothing else. No need to copy to individual variables, that is what you have the two pointers for.

And the complete credential_buffer is nul terminated, so you don't have to worry about the PSWD part, it automatically is.

I suggest that you read up on the C string functions. E.g strcpy and strncpy to copy nul terminated character arrays.

electrophile:
What I ended up doing is converting the array to a String and then back to a char array with the length of the string. Not sure if this is an efficient way to do it but it seems to be working.

The new problem now is that when I enter characters such as #, $, @ etc. they get converted into HEX values. For e.g. when I entered '#' the array now contains %23. Any way I can get the exact value entered?

It's called (to my knowledge) URL encoded.

The below code is an update of the earlier one and contains a function that handles the url decoding. For debugging, your on your own :wink: You can pass any C nul terminated string to it (not limited to the credential_buffer).
Note: I think that there are only two digits after the '%', so the below function only handles that.

#define CREDENTIALSIZE 64

// buffer with credentials
char credential_buffer[CREDENTIALSIZE] = "ssid=%23YZ&pswd=%24BC";

// pointers to the actual credential values in the buffer
char *accSSID;
char *accPWD;

/*
  url decode
  input: str; string to decode
  output: str; url decoded string
*/
void urlDecode(char *str)
{
  // buffer to store URL encoded hex values
  char hexbuffer[3];
  // clear hexbuffer
  memset(hexbuffer, 0, sizeof(hexbuffer));

  // pointer to percentage sign
  char *ptr;
  ptr = strchr(str, '%');
  while (ptr != NULL)
  {
    // read next two characters into hex buffer
    for (int cnt = 0; cnt < 2; cnt++)
    {
      hexbuffer[cnt] = ptr[cnt + 1];
    }
    // debug
    Serial.print("hexbuffer: "); Serial.println(hexbuffer);

    // convert to character (number) using bas 16
    char *endptr;
    long val = strtol(hexbuffer, &endptr, 16);
    if (endptr == hexbuffer)
    {
      // something wrong
      Serial.println("no digits");

      // point to next character in received credentials
      ptr++;
      // try to find next percentage
      ptr = strchr(ptr, '%');
      continue;
    }

    // and place in str (replaces '%')
    *ptr = (char)val;

    // move remainder 2 positions to left to get rid of the two digits
    memmove(ptr + 1, ptr + 3, strlen(ptr) - 2);
    // and place in str
    ptr = strchr(ptr, '%');
  }
}


void setup()
{

  Serial.begin(9600);

  Serial.print("Received credentials: "); Serial.println(credential_buffer);
  urlDecode(credential_buffer);
  Serial.print("Url decoded  credentials: "); Serial.println(credential_buffer);

  // check if credentials contain 'ssid='
  accSSID = strstr(credential_buffer, "ssid=");
  if (accSSID  == NULL)
  {
    // something wrong
    Serial.println("'ssid=' not found");
    // bail out
    return;
  }
  // increment accSSID so it points to the data
  accSSID += strlen("ssid=");

  // check if credentials contain '&pswd='
  accPWD = strstr(credential_buffer, "&pswd=");
  if (accPWD == NULL)
  {
    // something wrong
    Serial.println("'&pswd=' not found");
    // bail out
    return;
  }

  // accPWD points to '&'; replace by NUL character so SSID is null terminated
  *accPWD = '\0';

  // accPWD points to '&'; increment so it points to the data
  accPWD += strlen("&pswd=");

  // display result
  Serial.print("SSID: "); Serial.println(accSSID);
  Serial.print("PWD:  "); Serial.println(accPWD);

}

void loop()
{

}

sterretje:
Why would you. There is a reason why I renamed it to credential_buffer. It stores the credentials and nothing else. No need to copy to individual variables, that is what you have the two pointers for.

OK this then may seem like a dumb question (if till now they already aren't). The WiFi.begin function in Energia takes the following format: WiFi.begin(ssid, password) where ssid and password are character arrays and if I were to pass them the credentials buffer through pointers then would it be something like this:

WiFi.begin(*accSSID, *accPSWD)

Without the '*', yes.

sterretje:
Without the '*', yes.

Thanks! This helps a ton!