Strings larger than 256 bytes?

Is it possible to have strings larger than 256 bytes?

I would like to have something like longstring and not an array.

Thanks :slight_smile:

yes, you can implement a class that does uses long strings. Look at the code of the Arduino String class and with some minor tweaks it should be able to support longer strings.

Internally it still will be a char-array I guess, (or an array of char array's)

Problem with most string classes is (depends on implementation of course) that it fragments the memory because of continuous alloc/free actions.

Most of the applications that need long strings can be redesigned to work with smaller strings without the user noticing it.

Can you explain why your application needs long strings?

Arduino String can be larger than 256 characters. What are you doing that leads you to believe they can't?

Arduino String can be larger than 256 characters. What are you doing that leads you to believe they can't?

Keep in mind, though, that that one string is going to eat up 1/8 of the memory available on a UNO.

PaulS:

Arduino String can be larger than 256 characters. What are you doing that leads you to believe they can't?

Keep in mind, though, that that one string is going to eat up 1/8 of the memory available on a UNO.

That's true, but so would any other custom string class.

robtillaart:
Can you explain why your application needs long strings?

It's for my XELFA library that I mentioned earlier today in another post.
Basically what I need to do is to loop through a strings characters. As long as the string is 256 bytes long or less its ok,
but as soon as it exceeds 256 bytes the loop returns no character.
And thats when I'm using the brackets like: MyString
I'm experimenting a bit with it right now to see if I can find some simple code.

asically what I need to do is to loop through a strings characters.

Please post this code as there is something strange - Is the index used in the for loop an uint8_t aka byte?

A quick perusal of your library shows a few things...

You're allocating a char buffer of 256 bytes.

You have a bunch of static strings in your code. All these strings consume SRAM when the Arduino boots up, that's just the way the compiler handles them.

You're making heavy use of the String class, which can very quickly fragment the limited amount of SRAM available on the Uno.

One quick test to try out, eliminate or drastically cut down the size of your inData buffer. That alone is one quick method of freeing up SRAM. Then see what you String size limitation is. If it's larger than 256 bytes at that point, then you're just running out of SRAM.

Strings have the advantage that you quickly get a new copy.
How do you catch a bad_alloc exception?

Don't use String in situations where you don't know exactly what's happening.
( means: Use them only in situations when you could do as well without them )

michael_x:
Strings have the advantage that you quickly get a new copy.
How do you catch a bad_alloc exception?

Don't use String in situations where you don't know exactly what's happening.
( means: Use them only in situations when you could do as well without them )

To my knowledge, exceptions aren't available on the AVR implementation of C/C++ (likely due to the AVRs limited memory spaces). malloc failures result in a null buffer. Most of the String library operations silently abort on null buffers, but if you utilize the reserve method prior to said operations, it returns a zero or one based on the result of malloc. This would, of course, require you to determine how much memory is needed for the desired operation.

Ok I've narrowed down my problem.

From my experiments I can gladly say that theres no problem getting characters from long string.
You can either use brackets like String[](will return as a char), the String.substring() function or use String(String[]).
Any of them work fine.

And for the problem that I'm having I suggest you to visit the dedicated blog for my library so you can fully understand its string handling functions, visit xelfa.blogspot.com.

Ok, so XELFA have 2 functions that are used by the problem.
One function, the CountStr(), works fine by entering any size String.
However the CountListItems, which basically calls CountStr and results the same value as CountStr plus 1 (CountStr+1)
does not seem to receive the first parameter, unless the string is short.

Try this:
Take the following super long string: String MyValues = "00,0,0|100,26,0|200,28,0|300,29,0|400,30,0|500,32,0|600,31,0|700,32,0|800,34,0|900,37,0|1000,39,0|1100,41,0|1200,43,0|1300,46,0|1400,49,0|1500,52,0|1600,56,0|1700,60,0|1800,65,0|1900,71,0|2000,80,0|2100,88,0|2200,101,0|2300,114,0|2400,141,0|2500,180,0|2600,180,8|2700,180,18|2800,180,27|2900,180,36|3000,180,46|3100,180,57|3200,180,68|3300,180,77|3400,180,87|3500,180,97|3600,179,109|3700,155,109|3800,134,109|3900,117,109|4000,102,109|4100,88,109|4200,76,109|4300,66,109|4400,57,109|4500,50,109|4600,41,109|4700,35,109|4800,27,109|4900,21,109|5000,17,109|5100,10,109|5200,4,109|5300,0,115|5400,0,136|5500,0,170|5600,0,240|5700,-12,240|5800,-24,240|5900,-37,240|6000,-50,240|6100,-63,240|6200,-76,240|6300,-90,240|6400,-105,240|6500,-118,240|6600,-134,240|6700,-148,240|6800,-164,240|6900,-181,240|7000,-199,240|7100,-212,240|7200,-236,240|7300,-255,230|7400,-255,201|7500,-255,179|7600,-255,163|7700,-255,140|7800,-255,119|7900,-255,99|8000,-255,82|8100,-255,71|8200,-255,61|8300,-255,49"

And run this code:

Serial.println(String(XELFA.CountStr(MyValues,"|")));

Works like a charm, no matter how long the string is.

Now try to use the same string but with the following code:

Serial.println(String(XELFA.CountListItems(MyValues,"|")));

It wont work, unless the string is short.

The two functions look like this:

int XELFA::CountStr(String Str, String SubStr){
  //WORKING. DO NOT TOUCH!

  return FindStrPass(Str, SubStr, -1);
}


int XELFA::CountListItems(String List, String Spacer){
  int Count = CountStr(List, Spacer)+1;
  if (List=="") return 0;
  return Count;
}

Maybe I should use pointers instead of creating a bunch of new strings all the time?!

Btw I have an Arduino Mega.

If you're sure that CountStr is fine, the problem is perhaps in

(List == "")

which is not as trivial as it might look to you.

(List.length() == 0) might be a lot easier

String MyValues = "00,0,0|100,26,0 ... "

Serious?

This is where I would use an integer array, this is definitely not the way to program an Arduino imho.

Even if the data is send from a PC or sensor in this format, I would not store it in a string but convert it as fast as possible into an appropiate data array.

(Still I like the list functions, e.g. for Menu options)

robtillaart:
(Still I like the list functions, e.g. for Menu options)

... as a PROGMEM array of const char const * to PROGMEM ...

.. as a PROGMEM array of const char const * to PROGMEM ...

yep, exactly...

robtillaart:
Even if the data is send from a PC or sensor in this format, I would not store it in a string but convert it as fast as possible into an appropiate data array.

Nono, thats a string already pre-stored in my arduino.

This is the deal, those values in that string are the values of the two PWM pins used to control the RPM meter of my car.
The "|" character is a spacer, so if I were to use the GetListItem() function with that spacer and use index 1, I'd extract the following string: "100,26,0".
Now using a comma as a spacer and use the GetlistItem() function I could extract any of those three parameters, either "100" (represents 100 RPM's), "26" (represents PWM value for pin 12), or "0" (represents PWM value for pin 11).

I call that superlong string a mapper, because it has values that maps the values for my PWM pins.

NOW, I have 4 of those mappers, 1 for RPM, 1 for Speed in KM/h, 1 for Temperature, and 1 for Fuel Tank.

This could be fixed by using arrays, which I did use from the beginning, but then came the problem that when I made a function that finds the right index of of one of those parameters in one of those mappers, I had to use the same code 4 times to make it do the same thing basically.

The function I'm talking about is called int FindGaugeValueIndex(int Value, String Gauge);

and I want my code to look something like this (rough example):

int FindGaugeValueIndex(int Value, String Gauge){
  String GaugeTmp;
  if Gauge="RPM" then GaugeTmp=RPMValues;
  else
  if Gauge="Speed" then GaugeTmp=SpeedValues;
  else
  if Gauge="Temperature" then GaugeTmp=TemperatureValues;
  else
  if Gauge="Fuel" then GaugeTmp=FuelValues;

  for (int i=0; i <=CountListItems(GaugeTmp,"|")-1; i++){
    String Extracted = GetListItem(GaugeTmp,i,"|");

    int E_Value = XELFA.StrToInt(GetListItem(Extracted,0,","));

    if (Value>=E_Value) return i;
  }
}

What does this code tell us?
It's saying that if I want to get the PWM values of RPM position of lets say 2350 (we call this V),
it would first scan through the array of values, and see if V is equal to or higher than the current scanned value from our array (which isnt an array but it's a string instead) and we get returned with a proper value, since 2350 is not present in our array, we get returned with the PWM values of RPM 2300 instead. Kind of rounding down.

Next question is, why do I not use arrays?
because

  1. I have not found a way to do the same thing with arrays that you can do with strings, and that is to just say MyArray_A = MyArray_B.
    It wont work, values do not apply. therefore I would need 4 For-loop, which equals to more code, which in turn equals to more memory loss.
    Nontheless, the superlong string will eat memory wether its a string or an array.

but that's not the only reason why I'm not using arrays,
because using my technique will ensure that I could simply swap it out for something else at runtime.

etc etc.

I've been thinking alot about this and it is faster. Ive used the same technique in many projects before.

What does this code tell us?

It's a nice testbed for your String based library.

MyArray_A = MyArray_B

You can do that. MyArray_A should be a pointer, though. Or your Array is an object with a copy constructor, in which case you run into the same issues as with your String, just a bit later, as your binary array should be smaller than your String.

I have not found a way to do the same thing with arrays that you can do with strings

Bad luck for you. String is a wrapper around a char[], and hides that you easily do expensive stuff with innocent little operands.

int is a native data type for the ATMEl processor, String is not.
You can do everything with char and integers that you can do with String , and a lot more, and you are more aware of what you're doing.

Guess how many for loops are running whenever you do anything with a String.

ensure that I could simply swap it out for something else at runtime.

If that's a requirement, it rules out PROGMEM, and I agree it's simpler, if you do not have to fear buffer overruns.

michael_x:

You can do that. MyArray_A should be a pointer, though.

But I dont like pointers :frowning:

It's alright I'll figure it out sooner or later.

Thanks for all of your answers guys!