I will never understand Strings, strings, chars etc

Projects Programming
1

Hello all
Hope you all had a great Christmas and new year

After a madly frustrating Christmas trying to get my wireless project working, I think it's time I retired my (clearly fried and useless) 51 year old brain.

I have a complicated little setup of a Mega, talking wirelessly to a Pro Mini, talking over serial to another Mega.

The links are there, and we had a lengthy discussion on this issue before Christmas about formatting the communications (using the NRFl01 module library).

The data is sent using a String. I KNOW... Strings suck. But it's the only way it 'nearly' works.
If Strings cause memory issues, is there not a way to clear the String memory back to zero/clear after you have used it?

Why has nobody ever fixed the Arduino String feature?

This was because as the command is assembled, I used the simple += method.

But, all my attempts, and literally days of Googling how to change this system to an Array based affair have proven fruitless.
You guys did give me some demo code, but it didn't work and I have not been able to fix that.

My way around this, would be to use Teensy's, as the String function works on those.

I have no doubt there are people rolling their eyes and unable to understand why I can't change the method of my communication, but I am a hobbiest with clearly limited knowledge.

For instance.... this is my current issue:

Data is requested from my CNC machine.

String DataFromCNC;
String EditedDataFromCNC;
byte StartPos;

It arrives in a String as: "Echo data blah blah blah X:100 Y:100 Z:100 blah blah";

I then search that retrieved data to find the start of the co-ords:

StartPos=DataFromCNC.indexOf('X');

EditedDataFromCNC=DataFromCNC.substring(StartPos);

This all works. But it won't now let me make the original String DataFromCNC the contents of EditedDataFromCNC.
It needs to be put back into/as the original String for the Transmit routine to send it.

You can't + them together. strcpy fails with a 'cannot convert String to char' argument.
Eh? They are both Strings... where did the char come from.

Any link to an idiots guide to using Arrays might help.

I think me posting my code will be a car crash. 3 weeks of editing has resulted in a right mess.

2

have you considered transmitting a structure in binary instead?

create a structure with your components. write the structure to Serial, providing a pointer to the structure (probably cast as a byte*) and the sizeof the structure in bytes.

read that many bytes into a structure (cast as a byte*).

assumes they are the same endianess

you may need to encapsulate the data in a message with a header to delimit each message (type, length, data [])

3

C strings

C string is an array of chars. To build a C string you need an char array larger then the expected resulting string (at least 1 character more). The resulting string can be shorter then the array used to build it. To determine where the string ends a character with code 0 is used. To work with C strings a set of C functions is available. First useful is strlen. It runs over the array and counts the positions until the 0. So it returns the length of the string. Other functions can copy characters into char array from other char arrays, compare strings in zero terminated char arrays. Search for substrings in strings etc. All this functions know that the string ends with a 0.

Here is a list C - Strings

String

The String class wraps a C string char array. To allocate the array it doesn't know the resulting array size so it allocates some space in 'heap'. The String class has functions and operators to build the string. If needed it reallocates the internal char array to larger array and copies the characters from old array to new array. This reallocations fragment the heap memory. More here https://majenko.co.uk/blognode/4871

conversion

If some function takes const char* as parameter, you can pass the internal C string of the String object by accessing it with c_str() function. like ctrcpy(message, s.c_str()); where message is the target buffer (char array) for the copy

4

strcpy fails with a 'cannot convert String to char' argument.

I thought the String class had a method to return a C string?

5

The String is usable but in limited amount since the MCUs have limited memory amount. The best approach is to use constructor for flash memory String(const __FlashStringHelper *str) for constants. It is same as with normal strings to prevent copy constants into RAM for fix. The second "serious" rule is to use minimum String variables (I mean variable Strings) and make them with allocated size. Since I have my own implementation, I have defined constructor String(alloc_size_t N). This rule prevents memory re-allocation.

If Strings cause memory issues, is there not a way to clear the String memory back to zero/clear after you have used it?

It does, but here is a problem. If String about 4 characters length is created, it is unable to add next character. This is the reason why reallocation happens but allocation mechanism must find new free memory region in sufficient size. Old one is freed but this cause memory fragmentation and it happens each time when +, +=, = are used.

Why has nobody ever fixed the Arduino String feature?

This is not a problem with the String library, but a problem with how it is used and small memory.

You can't + them together. strcpy fails with a 'cannot convert String to char' argument.

The strcpy should work also but String must be converted to char string. There is a method c_str() for that. It returns char pointer to the String.

Any link to an idiots guide to using Arrays might help.

Arduino is just about C/C++. There are tons of guides and examples on the internet. Google!

6

SteveRC2017:
I have no doubt there are people rolling their eyes and unable to understand why I can't change the method of my communication

Just about there.

In my last post in your previous thread I provided a complete TX code for sending a struct with just a simple modification to @Robin2's tutorial example. I suggested that you put your specific project aside and just create an RX code (again with only a minor modification to @Robin2's tutorial RX example) to match my TX. What happened with that?

7

gfvalvo:
Just about there.

In [my last post in your previous thread
[/quote]
Which makes me wonder why the OP created another Thread about the same problem. I'm certainly not going to try to follow a discussion in two separate Threads.

...R](https://forum.arduino.cc/index.php?topic=654699.msg4414067#msg4414067)

8

One way i use to tame strings is to use the String methods that are available.

I create a String and pre-assign it memory space by using String.reserve(). Putting the reserved String to equal "" will clear the String buffer and to add to the String I use concatenate.

I have taken to using the ESP32 which has memory management features that clean up String memory fragmentation issues; through I still use String.reserve().

Here is how I am using Strings to receive Serial

void fReceiveSerial_LIDAR( void * parameters  )
{
  bool BeginSentence = false;
  char OneChar;
  char *str;
  str = (char *)ps_calloc(300, sizeof(char) ); // put str buffer into PSRAM
  for ( ;; )
  {
    EventBits_t xbit = xEventGroupWaitBits (eg, evtReceiveSerial_LIDAR, pdTRUE, pdTRUE, portMAX_DELAY);
    if ( LIDARSerial.available() >= 1 )
    {
      while ( LIDARSerial.available() )
      {
        OneChar = LIDARSerial.read();
        if ( BeginSentence )
        {
          if ( OneChar == '>')
          {
            if ( xSemaphoreTake( sema_ParseLIDAR_ReceivedSerial, xSemaphoreTicksToWait10 ) == pdTRUE )
            {
               xQueueOverwrite( xQ_LIDAR_Display_INFO, ( void * ) &str );
              xEventGroupSetBits( eg, evtParseLIDAR_ReceivedSerial );
              //
            }
            BeginSentence = false;
            break;
          }
          strncat( str, &OneChar, 1 );
        }
        else
        {
          if ( OneChar == '<' )
          {
            strcpy( str, ""); // clear string buffer
            BeginSentence = true; // found beginning of sentence
          }
        }
      } //  while ( LIDARSerial.available() )
    } //if ( LIDARSerial.available() >= 1 )
    xSemaphoreGive( sema_ReceiveSerial_LIDAR );
    //        log_i( "fReceiveSerial_LIDAR " );
    //        log_i(uxTaskGetStackHighWaterMark( NULL ));
  }
  free(str);
  vTaskDelete( NULL );
} //void fParseSerial( void * parameters  )
9

I have taken to using the ESP32 which has memory management features that clean up

Can you please provide a link to where these features are described ?

10

A: Character
(1) The following symbols/images of the English Language Alphabet are known as characters,
A B C ... Y Z
a b c ... y z
0 1 2 ... 9
, . ( ... !

(2) To store the characters of Step-1 into computer memory, there are 8-bit code (called ASCII Code) for each character and can be obtained from the following ASCII Code Table.

ascii.png
ascii.png599×502 111 KB

Figure-1:

(a) From Fig-1, we find that the ASCII code of A is 41 in hex base (= 65 in decimal base) and so on for other characters.

(3) To save A in a memory location, we make one of the following declarations:
char x = 'A'; //char is a keyword; x is an identifier
char x = 0x41;

ascii.png
ascii.png599×502 111 KB

11

UKHeliBob:
Can you please provide a link to where these features are described ?

Sure. https://esp32.com/

12

GolamMostafa:
(2) To store the characters of Step-1 into computer memory, there are 8-bit code (called ASCII Code) for each character and can be obtained from the following ASCII Code Table.

ASCII is a seven bit code.
You're thinking of extended ASCII, @ GolamMostafa.

13

Idahowalker:
Sure. https://esp32.com/

Thanks, but a link to the generic Espressif web site is less than helpful. I did track down the ESP32 Tech Ref manual but it is not helpful (to me at least) in explaining how its Memory Management Unit helps "clean up String memory fragmentation issues"

14

We have:
ASCII control characters (character code 0-31) which can be described by 5-bit on paper.
ASCII printable characters (character code 32-127) which can be described by 7-bit on paper.
The extended ASCII codes (character code 128-255) which can be described by 8-bit on paper.

In order to describe the above characters using memory locations, we need 8-bit code for each character -- this is the spirit that (mis)guides me to speak that a character has an 8-bit ASCII code.

15

UKHeliBob:
Thanks, but a link to the generic Espressif web site is less than helpful. I did track down the ESP32 Tech Ref manual but it is not helpful (to me at least) in explaining how its Memory Management Unit helps "clean up String memory fragmentation issues"

You are welcome.

One day, about a year ago, doing some searching on some item or other, on the ESP32 site, I ran into a conversation with one of the Espressif engineers. The explanation is when using freeRTOS, the OS of the ESP32, that loop(), assigned to core 1, is to be left empty. When loop(), under freeRTOS is ran, at the lowest priority, its function becomes house keeping, where loop() does heap de-fragmentation as one of its tasks.

16

That sounds interesting if a little worrying if the program running is time critical.

17

UKHeliBob:
That sounds interesting if a little worrying if the program running is time critical.

I don't think one would use an ESP32 (or ESP8266) for tasks that require very fine time control. Their strength lies in their WiFi system and WiFi is generally not a time critical activity.

...R

18

The ESP32 has a 64 bit timer for their own version of millis(). The ESP32 can detect and generate timer pulses to 12.5ns using one method. Using another method the ESP32 can generate pico second pulses, counts, times.
Example the PCNT API:
https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/peripherals/pcnt.html

19

Of course, nothing in the past dozen posts has been germane to the OP's problem.

20

gfvalvo:
Of course, nothing in the past dozen posts has been germane to the OP's problem.

so report to moderator :slight_smile: