The HATRED for String objects - "To String, or not to String"

cjdelphi:
with each iteration of the loop does a local variable get created (c/n) and recreated over and over again?

Yes. Automatic local variables are allocated on the stack and thrown away when the stack is unwound at the end of the call. In general, the more tightly you restrict the scope of the data the less chance there is for dependencies and unintended interactions between different parts of your code. This is why use of global data is best avoided where possible, and use of local data and arguments is generally preferred.

I think the String library initially allowed for that.

Paul Stoffgren's implementation did. The Arduino one removed that.

The option to allocate data in other-than-one-character larger increments needs to be part of the String class.

Typically, the user of the class has an idea of how much data is going to be stored in the String instance. Setting the minimum increment to some small value for small Strings and a larger value for larger Strings (say 5 and 15) would drastically reduce the amount of malloc/copy/free operations performed, which would have a big impact on the fragmentation issue. The default minimum increment should, of course, be 1.

PeterH:

cjdelphi:
with each iteration of the loop does a local variable get created (c/n) and recreated over and over again?

Yes. Automatic local variables are allocated on the stack and thrown away when the stack is unwound at the end of the call. In general, the more tightly you restrict the scope of the data the less chance there is for dependencies and unintended interactions between different parts of your code. This is why use of global data is best avoided where possible, and use of local data and arguments is generally preferred.

With 2k RAM I prefer globals and keeping control of my code. A few locals don't hurt but whole buffers in short often-called functions, it's good to be aware of because some day you might be writing code that needs to be fast rather than 'do it this way every time'.

awareness != mandatory practice

I've likened using C++ Strings on Arduino because C++ Strings are okay on a PC to putting a bathtub on a bicycle just because your house has one. I still feel the same. There's a lot of coding practices I've happily done on PC's that I wouldn't think of doing on Arduino.

GoForSmoke:

PeterH:

cjdelphi:
with each iteration of the loop does a local variable get created (c/n) and recreated over and over again?

Yes. Automatic local variables are allocated on the stack and thrown away when the stack is unwound at the end of the call. In general, the more tightly you restrict the scope of the data the less chance there is for dependencies and unintended interactions between different parts of your code. This is why use of global data is best avoided where possible, and use of local data and arguments is generally preferred.

With 2k RAM I prefer globals and keeping control of my code. A few locals don't hurt but whole buffers in short often-called functions, it's good to be aware of because some day you might be writing code that needs to be fast rather than 'do it this way every time'.

awareness != mandatory practice

I've likened using C++ Strings on Arduino because C++ Strings are okay on a PC to putting a bathtub on a bicycle just because your house has one. I still feel the same. There's a lot of coding practices I've happily done on PC's that I wouldn't think of doing on Arduino.

I too like the use of global variables in my arduino projects. I like that I can define them all at the top of the sketch with comments on what they represent. I kind of limit my use of local variables to things like the for statements and such. I know this is against the concept of good variable scoping practice that C/C++ encourages programmers to use and does make sense for larger projects where teams of programmers are working on the same application program and have to not step on each others coding functions, but with the limited SRAM space and me being the only one writing the code, global variables work better for me and I haven't heard of a good reason I should avoid them in my Arduino sketches. Sometime I just have to be an outlaw in heart I guess. :wink:

Lefty

Instead of removing strings, they just have to make about a 1-line change to the library code in free() that has a bug in it. Then some, at least, of the problems will go away.

So what is this one line change? Can it be posted so I can fix my library? I'd just rather fix things instead of hanger flying on the subject. Might eliminate some peoples constant string bashing and often mis diagnosis of problems people are having.

The problem is that the fix is not to source that is built during normal sketch compilation, but to the libraries provided pre-compiled in Arduino.

Have a look at this bug Google Code Archive - Long-term storage for Google Code Project Hosting.
and this thread: http://arduino.cc/forum/index.php/topic,95914.30.html

Personally, I am not a fan of String, even if some of the problems people run into are due to this bug. Anything that smells of dynamic allocation has no place, in my opinion, on such a memory-constrained platform. The problem is that statically allocated and strings and all the string.h stuff is complicated to a beginner. The String seems easy, so you can get something going faster than if you have to grok pointers and in-memory layout of strings and crap just to print hello world. But I think folks do not appreciate the limits of what you can do with String -- and with lots of other things like arrays and C strings too -- with so little memory, and wind up being forced to learn the details anyhow.

PaulS:
Part of the problem IS with the String class itself. When you want to append one character to an existing String, the length of the new String is the length of the old String plus the length of the String to append. That amount of space is allocated, and the old String is copied there, then the String to be appended is tacked on, then the old String's space is freed.

No this is not true. String uses realloc() which, if you'd care to read the implementation in realloc.c will always attempt an in-situ block-expansion. For most cases this will result in no pointer change at all and a simple adjustment of the allocated length - usually out into free space in most use-cases on a small system like this. A block-copy is only required if the original allocation either came from the free list and that space is now exhausted or there is a new block allocated in front of the one that needs to be expanded.

majenko:
Memory fragmentation occurs when small variables are removed from the heap to be replaced by larger variables. The space left by the small variable is too small to accept the larger variable, so the larger variable is instead placed on the top of the heap, causing the heap to grow. This most often occurs when working with strings and you want to add something to the end of a string (concatenate) or join two strings together.
to be shunned whenever possible.

It's actually quite hard to cause significant memory fragmentation to occur in the avr-libc implementation of the heap. free() will always coalesce blocks above and below the one it's being asked to release to create one larger block in the free list so even a poor String concat implementation that used malloc()+free() instead of realloc() would generally only end up with one block in the free list due to every second free() being coalesced with the previous one and then being big enough to satisfy the next malloc().

If you understand it, use it. If you don't, don't.

Assuming the String is at the top of the heap in 'most cases'. Otherwise not.

Andy Brown, I think you will find that Nick knows more about the code than you credit. And he isn't given to make unfounded claims.

So is there anything in the external programming that can be done to free the no longer used allocated memory space?

zoomkat:
So is there anything in the external programming that can be done to free the no longer used allocated memory space?

Perhaps enable the WDT, let if time out, and have the sketch start all over? :smiley:

Perhaps enable the WDT, let if time out, and have the sketch start all over?

Close to what my magic 8-Ball suggested. :slight_smile:

GoForSmoke:
Andy Brown, I think you will find that Nick knows more about the code than you credit. And he isn't given to make unfounded claims.

I don't have any argument with Andy. I think he has the fixed malloc/free library on his site anyway.

There is a bug in 'free' in the current Arduino library. I think we are all agreed on that. What Andy says about free spaces being combined by the library would almost certainly be correct. But the bug in question is something to do with the first (or last?) allocated block not being freed correctly, or something like that.

I have heard people say that, with the fixed free() installed, they can use String objects at some length with no problems, which supports what Andy says about the library combining unused blocks.

zoomkat:
So is there anything in the external programming that can be done to free the no longer used allocated memory space?

In one of the threads about this problem I suggested incorporating the corrected free function into existing code, and having a define along the lines of:

#define free myfree

in one of the header files that gets pulled into all libraries (including the String library). For the sake of a few dozen extra instructions you then have a workaround until the real library is fixed.

But the bug in question is something to do with the first (or last?) allocated block not being freed correctly, or something like that.

Then is there any way in the code to keep that particular block filled with nothing?

I took what Andy wrote to be saying there is no bug/problem. My error.

retrolefty:
global variables work better for me and I haven't heard of a good reason I should avoid them in my Arduino sketches.
Lefty

I can share at least one reason why you should reconsider globals. Here is a little sketch showing how a global can cause a very bad day.

This calculates the square of 2.

   //A global
   int i = 666;
 
   template <typename T> struct Base{
     Base( int i_NewVal ) : i( i_NewVal ) {return;}
     int i;
   };
   
   template <typename T> struct Squared : public Base<T>{
     Squared( int i_NewVal ) : Base<T>( i_NewVal * i_NewVal ) {return;}
     T Result() { return i; }
   }; 

  void setup()
    {
      Squared< long > s_Square( 2 );
      
      Serial.begin( 9600 );
      Serial.print( "2 squared = " );
      Serial.print( s_Square.Result() );
      return;
    }
    
  void loop(){return;}
   int i = 666;

I haven't even tried to run your sketch, but straight away, I sense trouble. :wink:

@pyro

You sketch is not about globals, rather it warns of the perils one faces by using poorly named variables :slight_smile:

The name of the variable could be something intuitive, I used 'i' for this example.
It does not affect the example. The sketch below works fine.

   //A global
   int i = 666;
 
  void setup()
    {
      int i = 2 * 2;
      
      Serial.begin( 9600 );
      Serial.print( "2 squared = " );
      Serial.print( i );
      return;
    }
    
  void loop(){return;}

In my previous example,
The global is a non-dependant name, it is satisfying a rule where non-dependant names in dependant types take the last definition seen. The compiler doesn't consider the base class during name lookup, even though its 'i' is the closer scope. This is quite different to the snippet above where the closer scope is setup::i which is also used.

So rather than receiving an error about 'i' being undefined in the class ( 'this->' is required to make the name dependant ), the previous definition of 'i' is silently used.

As a minimum, place the global in a namespace so it has to be explicitly used.