Reading data from web page and converting string to char array

Projects Programming
1

HI All,

I'm working on a project and i read a page or data page from an external web server, i got the data alright it seems ok BUT,
when i transfer the string received from the HTTPClient httpClient.getString() i got the data in infos ok but when i convert the data using
payload.toCharArray(charArray,strlength) ; the charArray is an char array define in the same function as where i receive the page data from httpclient, and strlength is equal the length of the string payload + 1 .
The problem is the data display ok when i print the string received, but when i transfer it to the charArray i get all garbage.

Ex:
251,#FFFFFF,251,0,0,1
252,#FFFFFF,252,0,0,1
253,#FFFFFF,253,0,0,1
254,#FFFFFF,254,0,0,1
255,#00FFFF,255,1,0,0
This is some part of the data received, and this is the part of charArray printed character by character.
'⸮?8⸮?⸮4⸮?⸮⸮⸮'⸮?8⸮?⸮,'⸮?3
not exactly like that but you get the picture.
I know the page was saved using the character set UTF-8, i change it to ANSI and saved the page again erasing the old coding from UTF-8 to ANSI.
Normally everything should be ok but..

Here is a small example of the code


void setPanelDataToObject(String payload) {
const char *lineSep = "\r\n" ;
  const char *delimiter = "," ;
  char *token ;
  char *p ;
  char *mystring ;
  int strlength = 0;
  Serial.println("Result of parsing for each lines") ;
  Serial.println("Should be complete lines with data each") ;
  strlength = payload.length() + 1 ;
      char charArray[strlength] ;
      Serial.println(payload) ;   // this print alwright
      Serial.print("Size of charArray") ;
      // that give me 11306 char element of 5662 bytes for the page

      int size = sizeof(charArray) / sizeof(charArray[1]) ;
          size += sizeof(charArray) / sizeof(charArray[0]) ;

      Serial.println(size);
      int i = 0;
      payload.toCharArray(charArray,strlength) ;
// print the charArray caracter by caracter
      while (i < size) {
        Serial.print((String)charArray[i]) ;
        i ++ ;
      }

I don't know if there is a character set specific for adruino or the C string that prevent the right caracter from being copy to the charArray

So if somewhone knows a trick to transfer correctly to the charArray to have the right caracter, i must be missing something here i'm stuck here can't go any furter on my code because of this
Please help someone !!!

2

what's this magic incantation supposed to do?

that's just no correct so when you do

      while (i < size) { ...

you get crap...

the size in bytes of the array is given by just sizeof. The count of elements in the array is given by sizeof(charArray) / sizeof(charArray[0]) 

void setPanelDataToObject(String& payload) { // <== pas by reference to avoid an extra copy
  char payloadCopy[payload.length()+1];
  strcpy(payload.c_str(), payloadCopy); 
  // now you have a copy as a cString
  ...
}
3

The sizeof is just for me it does not do anything in the code it give me and approximate idea of the file loaded string, the while i stuff it display the content of charArray data to serial port, this is not the crucial part of the code unless it directly affect the charArray date for wich i think it did not.
I'm more interested in the content of charArray values since i need to transfer the string to char array data.
Any idea ??

4

It does as you use it in the while loop to print the data and you go read beyond the buffer end which creates an undefined behavior situation for the code.

And that size does is just twice the size of the buffer, it does not give you any additional indication, it’s 2xstrlength

5

do you have non ASCII characters in your page? in UTF8 all ascii chars fit on 1 byte

also if you have large Strings and memory is short, the String duplication might fail silently

6

I know the character set was UTF-8 from the beginning, unfortunately i have no option to encode it in ASCII, and no i don't have specials character it is all numbers comma, dashes and thats it is all.
I thing the strintToCharacter array read the first byte of the character and since this is a control code from the encoding, that might be a reason why i have garbage caracter that may not be garbage at all but the first byte of the character that consist of 2 byte per each character. I don't have a converter that can take ASNI or UTF-8 and convert it ro ASCII.

Do you know a program that can do the convertion to ASCII, i thing that could solve one of the problem that i have otherwise i will have to create a program that scan the values and get only the significant byte of the character and try to convert it to single value for each value in ASCII 0-255 and 1 byte.

Let me know what you think Thanks.

7

Then your text is equivalent to ASCII.

Can you attach the file here?

8

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.