Optimising Code

I have the following code as part of a sketch. I would appreciate some help and/or suggestions on how to optimize it to reduce memory usage.

char Ref1[] = "1000";
char Ref2[] = "0001";
char Ref3[] = "A01";
char Ref4[] = "A";

String cid;
String fileName;

void setup()
  {

     // Compose file name
     cid = Ref1;
     cid += "-";
     cid += Ref2;             // 1000-0001
     fileName = cid;
     fileName += "_";
     fileName += Ref3;
     fileName += "_";
     fileName += Ref4;            // 1000-0001_A01_A
     fileName += ".txt";
    
   }

void loop()
  {
  }

First thing would be to ditch the String class and concatenate the char arrays yourself using strcat.

String cid;
String fileName;

Get rid of these. Strings waste memory.

You know how much room is needed for the file name. Allocate a char array one element larger, for the terminating null, and use strcpy() and strcat() to assemble the data.

Which part is variable ? You can use sprintf(), or even better sprintf_P() with PSTR, then the format string is in flash memory only.

int n = 1;
int m = 1;
sprintf(buffer, PSTR("1000-%04d_A%02d_A.txt"), n, m);

Why is reducing memory important ? Is this code for an Arduinon Uno or Leonardo or so. The Arduino SD library can only use filenames up to 8 characters. You filename is too long.

@Delta_G/PaulS : I know Strings are bad but was having a hard time figuring out how else to do it. Thanks for the pointers - I will try them out.

@Peter_n : Yes, the code if for a Leonardo. I switched from SD to SdFat library which if I understand supports long file names - but I might review the name.

The individual variables are needed to formulate a string (which I will now try to get rid of) for posting to an online database.

The concatenation of variables is also used as a displayed reference.

I am particularly interested in going from 1 as an integer to a padded format of 0001. I will test out your example - thanks.

The individual variables are needed to formulate a string (which I will now try to get rid of) for posting to an online database.

You do not want to get rid of strings. You want to get rid of Strings. Not the same thing at all.

PaulS: You do not want to get rid of strings. You want to get rid of Strings. Not the same thing at all.

Yes, I keep mixing up the terminology - but I understand what u mean, I should avoid declaring a variable as a String.

My new code :

int Ref1 = 39;
int Ref2 = 168;
char Ref3[] = "A01";
char Ref4[] = "A";

char cid[10];
char refList[15];
char fileName[20];

void setup()
  {

  // put your setup code here, to run once:
  Serial.begin(9600);
  while (!Serial) {;}
  //  sprintf_P(buffer, PSTR("0000-%04d_A%02d_A.txt"), n, m);
  
  sprintf_P(cid, PSTR("%04d-%04d"), Ref1, Ref2);
  Serial.println(cid);  // 0039-0068
  
  sprintf_P(refList, PSTR("%s_%s-%s"), cid, Ref3, Ref4);
  Serial.println(refList);  // 0039-0068_A01-A
  
  sprintf_P(fileName, PSTR("%s.txt"), refList);
  Serial.println(fileName);  // 0039-0068_A01-A.txt
}

void loop() {
  // put your main code here, to run repeatedly:
}

New code memory usage :

Sketch uses 3,534 bytes (10%) of program storage space. Maximum is 32,256 bytes. Global variables use 243 bytes (11%) of dynamic memory, leaving 1,805 bytes for local variables. Maximum is 2,048 bytes.

My old code :

char Ref1[] = "0039";
char Ref2[] = "0168";
char Ref3[] = "A01";
char Ref4[] = "A";

String cid;
String refList;
String fileName;

void setup()
  {
    Serial.begin(9600);
    while (!Serial) {;}

     // Compose file name
     cid = Ref1;
     cid += "-";
     cid += Ref2;             // 0039-0168
     Serial.println(cid);

     refList = cid;
     refList += "_";
     refList += Ref3;
     refList += "_";
     refList += Ref4;         // 0039-0168_A01_A
     Serial.println(refList);

     fileName = refList;
     fileName += ".txt";        // 0039-0168_A01_A
     Serial.println(fileName);
    
   }

void loop()
  {
  }

Old code memory usage :

Sketch uses 3,762 bytes (11%) of program storage space. Maximum is 32,256 bytes. Global variables use 236 bytes (11%) of dynamic memory, leaving 1,812 bytes for local variables. Maximum is 2,048 bytes.

Peter_n: You can use sprintf(), or even better sprintf_P() with PSTR, then the format string is in flash memory only.

I was expecting the new code to use more flash and less SRAM than the old code, but it appears the reverse is true.

I like this method because it gives me padding. Is any further improvement possible with this method?

I guess I should also compare using strcpy() and strcat() - need to go and learn how to use it first.

What are you using the final string for? Do you need to build it at all? (are you outputing
to a serial connection for instance?)

I should avoid declaring a variable as a string.

Haven't we covered this already. That statement is wrong. A String is a waste of resources. A string is a good thing.

PaulS: Haven't we covered this already. That statement is wrong. A String is a waste of resources. A string is a good thing.

The statement is correct. I should however, have capitalized the term.

I would appreciate some help and/or suggestions on how to optimize it to reduce memory usage.

If you want to optimise code don't use Arduino libs ! LOL It's fine to lash up some experimental code and to learn but optimised Arduino code using Arduino libs is an oxymoron.

To turn a LED on and off requires two writes to a h/w address, with digitalWrite it will take circa 4us at 16 MHz !

When you create a filename, you can make a buffer on the stack. If you use a seperate global buffer for every (temporary) use, you will use a lot memory.

If something is constant, use the 'const' keyword. The compiler can do a lot of funny things with that.

As a reference, I use your new code, compiled for Leonardo:

int Ref1 = 39;
int Ref2 = 168;
char Ref3[] = "A01";
char Ref4[] = "A";

char cid[10];
char refList[15];
char fileName[20];

void setup()
  {

  // put your setup code here, to run once:
  Serial.begin(9600);
  while (!Serial) {;}
  //  sprintf_P(buffer, PSTR("0000-%04d_A%02d_A.txt"), n, m);

  sprintf_P(cid, PSTR("%04d-%04d"), Ref1, Ref2);
  Serial.println(cid);  // 0039-0068

  sprintf_P(refList, PSTR("%s_%s-%s"), cid, Ref3, Ref4);
  Serial.println(refList);  // 0039-0068_A01-A

  sprintf_P(fileName, PSTR("%s.txt"), refList);
  Serial.println(fileName);  // 0039-0068_A01-A.txt
}

void loop() {
  // put your main code here, to run repeatedly:
}

6.068 bytes (21%) Maximum is 28.672 bytes. 224 bytes (8%) Maximum is 2.560 bytes

If a text is long, it can be placed in flash memory with PROGMEM, but Ref3 and Ref4 are short. The Ref1 and Ref2 will probably not be const in the final sketch, but this is just a test:

const int Ref1 = 39;
const int Ref2 = 168;
const char Ref3[] = "A01";
const char Ref4[] = "A";

void setup()
{
  char fileName[20];

  // put your setup code here, to run once:
  Serial.begin(9600);
  while (!Serial) {;}
  
  sprintf_P(fileName, PSTR("%04d-%04d_%s-%s.txt"), Ref1, Ref2, Ref3, Ref4);
  Serial.println(fileName);
}

void loop() {
  // put your main code here, to run repeatedly:
}

code size in flash : 6.010 bytes ram size : 175 bytes

MarkT: What are you using the final string for? Do you need to build it at all? (are you outputing to a serial connection for instance?)

The main reason for building it is the filename. On screen iterations could be multiple print manipulations, but of course each manipulation costs memory.

ardnut: If you want to optimise code don't use Arduino libs ! LOL It's fine to lash up some experimental code and to learn but optimised Arduino code using Arduino libs is an oxymoron.

C'mon now.. u cannot make a statement like that without at least throwing me a bone! Are u suggesting straight C++?

Peter_n: When you create a filename, you can make a buffer on the stack. If you use a seperate global buffer for every (temporary) use, you will use a lot memory.

If something is constant, use the 'const' keyword. The compiler can do a lot of funny things with that.

If a text is long, it can be placed in flash memory with PROGMEM, but Ref3 and Ref4 are short. The Ref1 and Ref2 will probably not be const in the final sketch, but this is just a test:

const char Ref3[] = "A01";
const char Ref4[] = "A";

Separate globals (Ref1 thru Ref4) are needed as they are used in their own right in the script.

I will have a long String global which is the string of data for posting to an online database. I would be interested to experiment putting that into flash mem using PROGMEM.

I note you declare char arrays as constants. I always thought "const" was only for use with integers. Seems I need to revise my thinking...

BTW the sketch is for a Leonardo (iBoardex), but the snippet testing I do on a spare Uno, hence the diff in max mem/ram values.

I will have a long String global which is the string of data for posting to an online database.

Nonsense. It will use SIGNIFICANTLY less memory to make the GET request in multiple packets. There is, of course, a tradeoff, in that using multiple packets takes a bit longer. In most cases, the tradeoff is well worth it.

In the rare cases when it is not, there is still no reason to piss away resources using the String class. The String class wraps a string. If you don't have memory for the string, you won't have memory for the wrapper. If you do, you certainly don't need the wrapper.

I would be interested to experiment putting that into flash mem using PROGMEM.

How do you propose to then write to that object, after you have placed the object in read-only memory?

I always thought "const" was only for use with integers. Seems I need to revise my thinking...

Yep.

@PaulS : Irrespective of what you may be thinking, my objective is actually not to waste resources ergo not to use Strings.

Unfortunately (for me) I am not competent enough yet to do so.

For example, below is a bit of code I was working on today and the only way I can get my desired results is to use Strings.

#include <EEPROM.h>

int ref[] = {0,0,1,1, 2,2,2,2, 3,3,3, 4,4};
int range[] = {1, 0,0,0, 0,0,8, 9, 0,0,0, 0,0,8};

int i;
int j = 0;
int mem;
int mem1 = 13;
int mem2 = 27;

String cid;

String a;
String b;
String c;
String d;
String e;
String f;

void setup() {
  Serial.begin(9600);
  while (!Serial);

  Serial.println("Ready...");
  
  for (i = 0; i < mem2; i++) {
    mem = EEPROM.read(i);
    Serial.print("Memory ");
    Serial.print(i);
    Serial.print(" = ");
    Serial.println(mem);
    cid += mem;
  }
  
  Serial.println(cid);
    
  for (i = 0; i < mem1; i++) {
    EEPROM.write(i, ref[i]);
  }
  
  for (i = mem1; i < mem2; i++) {
    EEPROM.write(i, range[j]);
    j++;
  }
  
  for (i = 0; i < mem2; i++) {
    mem = EEPROM.read(i);
    Serial.print("Memory ");
    Serial.print(i);
    Serial.print(" = ");
    Serial.println(mem);
    cid += mem;
  }
  a = cid.substring(0,4);
  b = cid.substring(4,8);
  c = cid.substring(8,11);
  d = cid.substring(11,13);
  e = cid.substring(13,20);
  f = cid.substring(20,27);
  
  Serial.println(a);	// 0011
  Serial.println(b);	// 2222
  Serial.println(c);	// 333
  Serial.println(d);	// 44
  Serial.println(e);	// 1000008
  Serial.println(f);	// 9000008
}


void loop() {
  // put your main code here, to run repeatedly:

}

So my question is, how can I do away with Strings in the above code.
The first step would to read the values from EEPROM and formulate “cid” as a non String - which I am having trouble to do.
If I achieve that, hopefully I can avoid the rest of the Strings.

There are a million ways to do this.
Here is my creation:

// The result can be a number of a complete string or substrings.
// I don't know what the code should be.
// So I decided to make a function to read the digits,
// and to be able to show it, and to split it into substrings.
// Tested on Arduino Uno

#include <EEPROM.h>

const byte ref[]   = {0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4};
const byte range[] = {1, 0, 0, 0, 0, 0, 8, 9, 0, 0, 0, 0, 0, 8};

const int mem1 = sizeof(ref);
const int mem2 = sizeof(ref) + sizeof(range);

char buffer[40];

void setup() {
  int i;
  
  Serial.begin(9600);
  while (!Serial);         // wait for leonardo.

  Serial.println("Ready...");

  FillBufferWithFormat(buffer, '.');
  Serial.println(buffer);
   
  for (i = 0; i < mem1; i++) {
    EEPROM.write(i, ref[i]);
  }

  int j=0;
  for (i = mem1; i < mem2; i++) {
    EEPROM.write(i, range[j]);
    j++;
  }
 
  FillBufferWithFormat(buffer, '.');
  Serial.println(buffer);

  FillBufferWithFormat(buffer, '\0');  // seperator is string terminator

  char *a = buffer + 0;
  char *b = buffer + 5;
  char *c = buffer + 10;
  char *d = buffer + 14;
  char *e = buffer + 17;
  char *f = buffer + 25;
  
  Serial.println(a);  // 0011
  Serial.println(b);  // 2222
  Serial.println(c);  // 333
  Serial.println(d);  // 44
  Serial.println(e);  // 1000008
  Serial.println(f);  // 9000008
}


void loop() {
  // put your main code here, to run repeatedly:
}

void FillBufferWithFormat( char *buf, char seperator)
{
  // Format: XXXX.XXXX.XXX.XX.XXXXXXX.XXXXXXX
  //         01234567890123456789012345678901234
  //         0         1         2         3
  int j = 0;                     // buffer index
  for (int i = 0; i < mem2; i++) 
  {
    buf[j++] = '0' + EEPROM.read(i);    // make character of digital number
    if( j==4 || j==9 || j==13 || j==16 || j==24)
    {
      buf[j++] = seperator;
    }
  }
  buf[j] ='\0';        // terminate string
}

@Peter_n : Thank-you. You have earned some well deserved karma points from me.

There may be a million ways - but so far I only figured out one and it is way more crude than yours.

Before I received your reply did manage to figure out how to read from EEPROM into an integer array.
To derive the first variable “a” I did this :

for (i = 0; i < 4; i++) {
    int mem = EEPROM.read(i);
    mema[i] = mem;
  }

/*  int mema2 = mema[2]*10 + mema[3];
    mema2 = mema[1]*100 + mema2;
    mema2 = mema[0]*1000 + mema2;
*/  
  
  int mema2 = mema[0]*1000 + mema[1]*100 + mema[2]*10 + mema[3];

  sprintf(mema3, ("%04d"), mema2);
  
  Serial.println(mema3);    // 0011

Ugly I know and the conversion to a concatenated value is a lot more memory hungry than yours - but at least I advanced my knowledge :slight_smile:

I will study your eloquent solution and learn from it.
Thank-you again.

That's not ugly at all. Make numbers of it, using the "weight" of the digit (1, 10, 100 and 1000) and use sprintf. That's very common programming. I give it 10 points out of 10.

The EEPROM.read() and EEPROM.write() are Arduino functions. The gcc-avr library has function to read and write a buffer. There is even an update function, which skips the byte that have already their value. http://www.nongnu.org/avr-libc/user-manual/group__avr__eeprom.html The eeprom_write_block() and eeprom_update_block() might be useful.

I give it 10 points out of 10.

Can't be more than a 9, because OP is storing the byte returned by EEPROM.read() in an int. That shows a lack of understanding.

PaulS:
Can’t be more than a 9, because OP is storing the byte returned by EEPROM.read() in an int. That shows a lack of understanding.

Ah yes - but I am learning … :slight_smile:
I now know how to change that integer to a character without sprintf - thanks to Peter_n’s code.

However, I am a little stuck.

I would like to initialize the variables directly with an EEPROM read - this is after ref and range have been stored in EEPROM.

I have come up with this code which works.

 #include <EEPROM.h>
 
 // int ref[] = {1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4};
 //    memloc    0  1  2  3  4  5  6  7  8  9  10 11 12
 //    var       a0 a1 a2 a3 b0 b1 b2 b3 c0 c2 c2 d0 d1
 
 // int range[] = {5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6};
 //    memloc      13 14 15 16 17 18 19 20 21 23 24 25 26 27
 //    var         e0 e1 e2 e3 e4 e5 e6 f0 f1 f2 f3 f4 f5 f6

char a[5];
char b[5];
char c[4];
char d[3];
char e[8];
char f[8];

const byte memloc_a = 0;
const byte memloc_b = 4;
const byte memloc_c = 8;
const byte memloc_d = 11;
const byte memloc_e = 13;
const byte memloc_f = 20;
 
void setup() {
   Serial.begin(9600);
   while (!Serial);
   Serial.println(F("Serial Ready..."));
 
   int mem;
   int j = 0;
   for (int i = memloc_a; i < (sizeof(a) - 1); i++) {
   mem = EEPROM.read(i);
   a[j] = '0' + mem;
   j++;
   }
   Serial.println(a);

  }
  
 void loop() {
   }

So now I obviously need to turn that bit of code into a function.
So far I have this code which does not compile :(.
From the error messages there is a problem with the function code itself and how I am trying to invoke the function.

 #include <EEPROM.h>
 
 // int ref[] = {1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4};
 //   memloc    0  1  2  3  4  5  6  7  8  9  10 11 12
 //   var       a0 a1 a2 a3 b0 b1 b2 b3 c0 c2 c2 d0 d1
 
 // int range[] = {5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6};
 //   memloc      13 14 15 16 17 18 19 20 21 23 24 25 26 27
 //   var         e0 e1 e2 e3 e4 e5 e6 f0 f1 f2 f3 f4 f5 f6

char a[5];
char b[5];
char c[4];
char d[3];
char e[8];
char f[8];

const byte memloc_a = 0;
const byte memloc_b = 4;
const byte memloc_c = 8;
const byte memloc_d = 11;
const byte memloc_e = 13;
const byte memloc_f = 20;
 
void setup() {
   Serial.begin(9600);
   while (!Serial);
   Serial.println(F("Serial Ready..."));

/* 
   int mem;
   int j = 0;
   for (int i = memloc_a; i < (sizeof(a) - 1); i++) {
   mem = EEPROM.read(i);
   a[j] = '0' + mem;
   j++;
   }
   Serial.println(a);
*/

  }
  
char ExtractEepromVariable( a, memloc_a);
  
  Serial.println(a);
  
 void loop() {
   }
   
 char ExtractEepromVariable( char var[], byte memloc) {
   int mem;
   int j = 0;
   for (int i = memloc; i < (sizeof(var) - 1); i++) {
   mem = EEPROM.read(i);
   var[j] = '0' + mem;
   j++;
   }
return var;
 }

Normally (in PHP) I would create a new variable, something like :

mvVariable = ExtractEepromVariable( a, memloc_a) ;

But this is not PHP so some help would be appreciated.