Reading from Serial with unknown length

Hello,

I need to read in characters from Serial, but I don't know how many characters there will be, or what they will be. Everything I've seen and read says that actual Strings are a waste of resources, although they let me do what I need to do quite easily. I also know that I could read the Serial data into a String, then use toCharArray to convert it to a char array, but I've seen people say that's not a good idea either.

The fact that everyone says Strings are a waste of resources along with the fact that I need to write these strings to an SD card, which requires a char array, means that I'm trying to write my code using char arrays instead.

I've found plenty of examples where the maximum length of the input string is known ahead of time. They define the char array with a maximum size, and read in from Serial character by character, making sure to not exceed the maximum size - 1 (to leave room for the null).

But again, I don't know ahead of time how big the data is I'll be getting in. Sometimes it could be as little as four characters, other times it could be hundreds.

For context, what I'm trying to do is create a keyboard macro device. It has the macros stored as text files on the SD card, and it has a keypad that I can press to activate the macros. When I press a button, it reads the corresponding file from the SD card and sends the keyboard macro to the computer.

I don't want to have to take the SD card out every time I need to edit a macro, so I want to be able to pull up a serial port, connect to the device, and issue commands like "MODIFY A1" to modify macro A1. Then I would start typing out the macro, and that's where the length of the lines is unknown.

If anyone can tell me how to accomplish this, I'd really appreciate it.

Rick

There is a very handy, efficient, well documented and extremely flexible library <string.h> for handling C-strings, which are zero terminated character arrays.

Unlike Strings, C-strings work reliably on a standard Arduino.

You might find this introduction to reading the Arduino serial port helpful.

Sometimes it could be as little as four characters, other times it could be hundreds.

You will have to use good sense handling the input. There is no reason AT ALL to read in hundreds of characters before doing something with them, and too little memory to squander on anticipating the worst case.

rickseiden:
Then I would start typing out the macro, and that's where the length of the lines is unknown.

If it is a communication between a man and a machine in this world, then there is a 'protocol' to which both parties remain agreed for the contracted time period. Your idea/proposal for a data with unknown length does violate the 'well established' protocol concept. The data sender must somehow let the receiver know in advance about the nature of the data so that the receiver can reliably accept it -- the length of the data or the 'start mark' and 'end mark' of data.

rickseiden:
I don't want to have to take the SD card out every time I need to edit a macro, so I want to be able to pull up a serial port, connect to the device, and issue commands like "MODIFY A1" to modify macro A1. Then I would start typing out the macro, and that's where the length of the lines is unknown.

In this case, there's no need to worry about the maximum length because you don't need to store the entire thing in a string. What you should do is read some data from Serial, write it to SD, then repeat that until there's no more data available from Serial.

pert:
[...] until there's no more data available from Serial.

That means that there is a 'mark' that says that there is no more data.

I’m with pert on this one.

If you don’t know the length of the data, select a buffer of reasonable size and track how much you have saved into it. When the buffer is full, write it out to a temporary file on the SD card and reuse the buffer. Repeat until you have the whole data sent (how do you know this?), spooled to the file on SD card. You can the use the SD card file size functions to dynamically allocate a new buffer of the right size if you need to read it back into into memory to do something with it.

I need to read in characters from Serial, but I don't know how many characters there will be, or what they will be. Everything I've seen and read says that actual Strings are a waste of resources, although they let me do what I need to do quite easily.

How would you do it using Strings ?

You are presumably referring to the Serial.readString() or Serial.readStringUntil() functions. If so, then there is no reason why you should not write the equivalent functions to read Serial input and save it in a string

UKHeliBob:
there is no reason why you should not write the equivalent functions to read Serial input and save it in a string

No need to write one. They already exist:

Even better