Best way to "run multiple sketches"?

So I am making a multi-use led matrix tool, which will have multiple "apps" in it, for example, a music sequencer, accessible through a menu.

I already have a separate sketch that is the working sequencer, but I don't know exactly how to make this part of a bigger sketch.

Normally, in a situation like this, I would obviously have a function for each "app", but I would like to free the arduino's memory after the app is finished running, that is, I planned on making the variables used in the function local, so once the function completes, the memory (RAM?) can be used for other things. But there are functions declared in the original sequencer sketch, and I can't declare these within the sequencer function, nor can I declare them outside the sequencer function, since some variables would be not in the scope of these functions. (they would be local inside the sequencer function)

I also do not want to have to make major changes to the original sequencer sketch, if I don't have to. It's kind of complex (and probably a bit kludgey) and I do not want to have to rewrite it. (if it ain't broke, don't fix it, as the cliched phrase goes...)

I was thinking of making a class for the sequencer, it would have a public function that would run the code from the original sequencer sketch, and private variables and other functions. When I "quit" the sequencer "app", couldn't I destruct the instance of the class and free the arduino's memory up from those variables?

Would this work? If not, can you guys suggest a better option?

Thanks! (and sorry for this wordy post)

I saw a very interesting way of doing this not to long ago- It uses goto statements, labels and infinate loops.

Using a label is usually a big NO-NO in C programming, but I have to say, it looked good!

void setup() {//setup whatever}

void loop() {

label1:
while(1) {
//define locals
//mini-program
//if finished
goto label2;
}

label2:
while(1) {
//define locals
//mini-program
//if finished
goto label1;
}

}

I haven't tried to use it yet, so tell me how it goes!

@InvalidApple - how does that get around the RAM deallocation problem?

Okay, I think that may be the simplest solution.
And it shouldn't be too much of a no-no, since there will be not too many gotos and only one goto per label.

So the syntax is the name of the label, then a colon?

I just ask because I have never used gotos in a program. (probably a good thing)

And I think it would get around the ram deallocation problem, if the while loop is inside a function, making the variables local to that function.

If anyone has other suggestions, I'm open to other ideas!

If the individual sketches you want to combine each uses global variables that are not needed by the other sketches, you can define them within a union:

union {
  byte bigarray1[100];  // needed by one sketch
  word bigarray2[50];   // needed by another sketch
} arrays;

That way they are stored on the same memory locations, so you don't need room for both.

Hmm I don't really see how that would help.
I looked online, and it said only one object in a union is stored at any time.
If I understand correctly, that means I would have to make a union for every single global variable in the original sketch.

union can be combined with structs to get around that:

union {
    struct {
        byte foo;
        int bar;
        byte bigarray[100];
    } sketch1;

    struct {
        long baz;
        int bab;
        int bigarray[50];
    } sketch2;
} globals;

I would like to free the arduino's memory after the app is finished running

I'm not sure what your dilemma is. From the main() program you are running one function (based on a sketch) at a time, right?

Well...

Storage for local non-static variables, class objects, etc., is allocated when the function is invoked (i.e. when it is called from another function), not when it is declared. When code returns from a function, the local non-static variables, class objects, etc., go "out of scope," and the memory is de-allocated. Automatically. (That's why they are called "automatic" variables.)

Of course if a function calls another function, the storage for the calling function remains in place and the called function allocates (and de-allocates) storage for its automatic variables as required.

An Operating System can swap programs (and pieces of programs) into and out of system RAM from external storage (disk files, for example) and give each of several processes a time slice of a few milliseconds before going to the next one. In that way there is an appearance of concurrency even though not all of the processes are actually running at the same time. (Of course when it does the swap to the next process, it saves the entire context of the previous process so that it can take up where it left off when its time comes again.)

You could make your own scheduler that transfers control to multiple "processes" one at a time, but there is nothing to "swap" them out with.

For programs running under operating systems (Windows, Linux, etc.) static variables (globals and locally-declared "static" variables) come from the system ("heap") and are not part of the "stack," but for standalone systems like stuff running on a 8-bit ATmega, there is no "system" to get "heap" memory from. Constants can be stored in program memory rather than RAM, but there's precious little else you can do.

Bottom line:
The total RAM required at any given time is equal to the sum of RAM required for all functions that are active at a given time (plus overhead for storing arguments and return addresses on the program stack). Storage for static variables (global or "file-scope" variables) remains throughout the run of the main() program.

Regards,

Dave

Footnote: Making more and more variables "global" may save some stack space by not having to pass parameters when a function is called, but doesn't really save much unless you are at the very limit of memory (in which case you might have to resort to assembly language programming to make absolutely that nasty old compiler doesn't waste precious memory space---lot's of luck trying to out-optimize avr-gcc!)

And I think it would get around the ram deallocation problem, if the while loop is inside a function, making the variables local to that function.

I don't think it works that way...you might want to look at this:

http://www.nongnu.org/avr-libc/user-manual/malloc.html

Regardless, it probably won't be easy...

[edit]Hmm - this comment seems to say that you are correct, though - so I am probably wrong:

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1278600656/2#2

:)[/edit]

Thanks for the help, but I think I can get around not having memory deallocated after an "app" finishes.

I've realized that some of the code is a bit kludgeish, so that some variables are unnecessary. Also, some variables will be able to be used by multiple apps at different times. Finally, there relatively are not that many variables in play.

In other words, I think that 2kb of ram will be sufficient even if I declare all variables globally.

And Dave, my problem was that I cannot simply copy and paste a working sketch inside a function (ignoring obvious errors such as having multiple setups and loops). I could put variables inside the function, and so the memory would be deallocated, but any function definitions from the original sketch cannot be within the main function. Nor can they be outside because many use variables that are not passed to them as parameters, and these variables would be inside the main function, causing compiler issues with the other function definitions.

I cannot simply copy and paste a working sketch inside a function

Well, I'm not sure exactly what you have in mind that you can't do, so I'll show you what I had in mind that I can do.

If it doesn't suit your style or meet your needs, then I'm sorry for wasting time and bandwidth.

I will make two sketches, each of which takes up more than 1K bytes of RAM. Since the sketches will be used as "applet" functions, for testing them, I will make them exit at the end of loop() rather than repeating endlessly as the Arduino main() function normally dictates. In other words, the "loop" is one-shot and done.

I will test them thouroughly.

Then I will make a "master" sketch whose loop repeatedly calls two functions based on the two sketches. The two "applet" functions and the controlling sketch are, perhaps, overly simplistic, but I just want to show what could be done without much editing of the tested sketches. (And with nary a dreaded "goto" or any of those dang superfluous global variables in sight!)

Note that any RAM taken by global variables or by local variables in the "master" sketch is still in place when the functions are called, but the massive (for an ATmega328) amounts for each function are automatically released as each function returns. (Which was kind of the point.)

Anyhow...

Here's the first sketch:

//Sketch1
void setup()
{
  Serial.begin(9600);
}

void loop()
{
  long int x[250] = {0};
  char buffer[256];
  x[249] = 42;
  sprintf(buffer, "sizeof(x) = %d, sizeof(buffer) = %d\n",
                   sizeof(x), sizeof(buffer));
  Serial.print(buffer);
  Serial.println(millis());
  sprintf(buffer, "In sketch1: x[249] = %d at ", x[249]);
  delay(1000);
  Serial.println("End of loop() in sketch1");
  Serial.println();
  //
  // The Arduino way is that a main() function runs and reruns the loop continuously.
  // For our purposes, we want the "sketch" to quit after it does its business
  // one time.  In general, there may be some amount of logic that controls when the
  // "sketch" is really through.
  exit(0);
}

Here's the second:

//Sketch2
void setup()
{
  Serial.begin(9600);
}


void loop()
{
  long int x[250] = {0};
  char buffer[256];
  x[249] = -42;
  sprintf(buffer, "sizeof(x) = %d, sizeof(buffer) = %d\n",
                   sizeof(x), sizeof(buffer));
  Serial.print(buffer);
  sprintf(buffer, "In sketch2: x[249] = %d at ", x[249]);
  Serial.print(buffer);
  Serial.println(millis());
  
  delay(1000);
  Serial.println("End of loop() in sketch2");
  Serial.println();
  // The Arduino way is that a main() function runs and reruns the loop continously.
  // For our purposes, we want the "sketch" to quit after it does its business
  // one time.  In general, there may be some logic that controls when the
  // "sketch" is really through.
  exit(0);
}

Here's the "master" sketch with the two functions made from the two "applet" sketches:

//
// Main control function: sketchmaster
// This sketch calls functions made from previously
// tested sketch1 and sketch2
//
// Serial is a (global) instance of the HardwareSerial
// class, instantiated in HardwareSerial.cpp, and
// is linked automatically when you build from
// within the Arduino IDE.
//
// All other variables are local (automatic) within their
// functions.
//
void setup()
{
  Serial.begin(9600);
}
//
// Of course you could have lots of conditionals and other
// logic to determine which function to call and when.
//
// For this simple case, we simply call them in direct rotation.
//
void loop()
{
  Serial.println("main loop calling function1()");
  function1();
  
  Serial.println("main loop calling function2()");
  function2();
}

//
// Was loop() in Sketch2
//
// Note that the two arrays take up a total
// of 1256 bytes (over half of the total 2K
// of the ATmega328p).  If sketch2 had a setup()
// that didn't involve Serial or other main function
// globals, put its setup() stuff at the beginning
// of function2
//
void function2()
{
  long int x[250] = {0};
  char buffer[256];
  x[249] = -42;
  sprintf(buffer, "sizeof(x) = %d, sizeof(buffer) = %d\n",
                   sizeof(x), sizeof(buffer));
  Serial.print(buffer);
  sprintf(buffer, "In sketch2: x[249] = %d at ", x[249]);
  Serial.print(buffer);
  Serial.println(millis());
  
  delay(1000);
  Serial.println("End of loop in sketch2");
  Serial.println();
  // The Arduino way is that a main() function runs and reruns the loop continuously.
  // For our purposes, we want the "sketch" to quit after it does its business
  // one time.  In general, there may be some logic that controls when the
  // "sketch" is really through.
  // Since this is now a function, we merely return, not exit()
  // exit(0);
  
  // A void function can return by merely dropping off of the end.
}

//
//Was loop() in Sketch1
//
// Note that the two arrays take up a total
// of 1256 bytes (over half of the total 2K
// of the ATmega328p).  If sketch1 had a setup()
// that didn't involve Serial or other main function
// globals, put its setup() stuff at the beginning
// of function1()
//
void function1()
{
  long int x[250] = {0};
  char buffer[256];
  x[249] = 42;
  
  sprintf(buffer, "sizeof(x) = %d, sizeof(buffer) = %d\n",
                   sizeof(x), sizeof(buffer));
  Serial.print(buffer);
  
  sprintf(buffer, "In sketch1: x[249] = %d at ", x[249]);
  Serial.print(buffer);
  Serial.println(millis());
  
  delay(1000);
  Serial.println("End of loop in sketch1");
  Serial.println();
  // The Arduino way is that a main() function runs and reruns the loop continously.
  // For our purposes, we want the "sketch" to quit after it does its business
  // one time.  In general, there may be some logic that controls when the
  // "sketch" is really through.
  // Since this is now a function, we merely return, not exit()
  // exit(0);
  // A void function can return by merely dropping off of the end.
}
//
//Was loop() in Sketch1
//
// Note that the two arrays take up a total
// of 1256 bytes (over half of the total 2K
// of the ATmega328p).  If sketch1 had a setup()
// that didn't involve Serial or other main function
// globals, put its setup() stuff at the beginning
// of function1()
//
void function1()
{
  long int x[250] = {0};
  unsigned timeNow;
  char buffer[256];
  x[249] = 42;
  sprintf(buffer, "sizeof(x) = %d, sizeof(buffer) = %d\n",
                   sizeof(x), sizeof(buffer));
  Serial.print(buffer);
  timeNow = millis();
  sprintf(buffer, "In sketch1: x[249] = %d at ", x[249]);
  Serial.print(buffer);
  Serial.println(millis());
  delay(1000);
  Serial.println("End of loop in sketch1");
  Serial.println();
  // The Arduino way is that a main() function runs and reruns the loop continuously.
  // For our purposes, we want the "sketch" to quit after it does its business
  // one time.  In general, there may be some logic that controls when the
  // "sketch" is really through.
  // Since this is now a function, we merely return, not exit()
  // exit(0);
  // A void function can return by merely dropping off of the end.
}

Output of the master:

main loop calling function1()
sizeof(x) = 1000, sizeof(buffer) = 256
In sketch1: x[249] = 42 at 98
End of loop in sketch1

main loop calling function2()
sizeof(x) = 1000, sizeof(buffer) = 256
In sketch2: x[249] = -42 at 1229
End of loop in sketch2

main loop calling function1()
sizeof(x) = 1000, sizeof(buffer) = 256
In sketch1: x[249] = 42 at 2362
End of loop in sketch1

main loop calling function2()
sizeof(x) = 1000, sizeof(buffer) = 256
In sketch2: x[249] = -42 at 3496

.
.
.

Regards,

Dave

Footnote: Instead of putting the source code for the functions in the master sketch, you could, of course, put the functions (together or separately) into a library (or libraries) and import the library (or libraries) into your master sketch. Instead of functions, they could be made into classes, and your master sketch could instantiate objects in sequence in its loop() function. Whatever...

The title that you gave the thread is "Best way..." I make no claims of best, but it is a way.

Instead of functions, they could be made into classes, and your master sketch could instantiate objects in sequence in its loop() function. Whatever...

Exactly what I had thought of in the OP. And just for clarification, a class initiated within a function gets uninitiated when the function completes, like any other datatype, right?

And as to your view on the solution, say I had a sketch like this:
Leds are on pins 1-8. It will blink random combinations of lit leds.
(not tested code)

int variable = 500;
byte quack;
int array[] = {
  1, 2, 3, 4, 5, 6, 7, 8};

void setup()  {
  setup stuff
}

void loop()  {
  for(int i = 0; i<=7; i++)  {
    int rand = random(0,1);
    if(rand) == 1  {
      digitalWrite(array[i], HIGH);
    }
    else {
      digitalWrite(array[i], LOW);
    }
  }
  delay(500);
  allOff(variable);
}

void allOff(int duration)  {
  for(int i = 0; i<=7; i++)  {
    digitalWrite(array[i], LOW);
  }
  delay(duration);
}

I'm not going to write a second "sketch", just imagine there is one, where the following function is called in loop().
So I'll make a function out of that sketch:

void randomBlink()  {

  int variable = 500;
  byte quack;
  int array[] = {
    1, 2, 3, 4, 5, 6, 7, 8};

  for(int i = 0; i<=7; i++)  {
    int rand = random(0,1);
    if(rand) == 1  {
      digitalWrite(array[i], HIGH);
    }
    else {
      digitalWrite(array[i], LOW);
    }
  }
  delay(500);
  allOff(variable);


  void allOff(int duration)  {
    for(int i = 0; i<=7; i++)  {
      digitalWrite(array[i], LOW);
    }
    delay(duration);
  }

}

Hmm, can't do that. The function definition is inside the other function definition, no,no,no.
I'll put the void allOff(int duration) outside randomBlink()

void randomBlink()  {

  int variable = 500;
  byte quack;
  int array[] = {
    1, 2, 3, 4, 5, 6, 7, 8  };

  for(int i = 0; i<=7; i++)  {
    int rand = random(0,1);
    if(rand) == 1  {
      digitalWrite(array[i], HIGH);
    }
    else {
      digitalWrite(array[i], LOW);
    }
  }
  delay(500);
  allOff(variable);

}

void allOff(int duration)  {
  for(int i = 0; i<=7; i++)  {
    digitalWrite(array[i], LOW);
  }
  delay(duration);
}

Can't do that either, array[] is not within the scope of that function definition. I would put it as a parameter in this case, but one function in my original sketch uses 14 variables, not worth the trouble of making 14 parameters to that function.

So, your solution is unfortunately not sufficient. (if I understand my own situation correctly)

So unions work with Arduino or its compiler after all. I thought it was said in here Arduino forums that they wont work.

...a class initiated within a function gets uninitiated when the function completes, like any other data type, right?

Yes. Well, more precisely, whenever an object goes out of scope its destructor is called.. Generally speaking, that means that the memory that was occupied by that object is available whenever the program needs memory. That is certainly true for local variables (including class objects) inside a function. With avr-gcc and other GNU compilers that I have tested, it is also true for variables that are local within any program block, provided that you use the "-Os" command line switch (which Arduino helpfully provides for us).

int variable = 500;

byte quack;
int array[] = {
 1, 2, 3, 4, 5, 6, 7, 8};

void setup()  {
 setup stuff
}

void loop()  {



Hmm, can't do that...

You know the solution already: Get rid of those annoying global variables and pass parameters to functions.

// A sketch without any of those annoying global variables
// that make it difficult to convert to a function for
// a stand-alone "applet"

void setup()
{
    // setup stuff
}

void loop()
{

    int variable = 500; // Use this somewhere, I guess
    byte quack; // Use this somewhere, I guess

    // Local variable will be used as an argument
    int array[] = { 1, 2, 3, 4, 5, 6, 7, 8 };

    for (int i = 0; i <= 7; i++) {
        int rand = random(0, 1);
        if (rand == 1) {
            digitalWrite(array[i], HIGH);
            }
        else {
            digitalWrite(array[i], LOW);
        }
    }
    delay(500);
    allOff(array, variable);
}

void allOff(int array[], int duration)
{
    for (int i = 0; i < 8; i++) {
        digitalWrite(array[i], LOW);
    }
    delay(duration);
}

Now functionalizing the sketch is easy, right?

not worth the trouble of making 14 parameters to that function.

Non-Arduino C and C++ programmers usually learn (or have it pounded into their little peanut heads) to avoid unnecessary globals (for lots of reasons). Arduino beginners with no good programming experience tend to do things the "easy way," with globals, but maybe for applications like yours, it's time to "get over it." I mean, BASIC programmers in the 1970's had no choice: everything was global. Nowadays, we have a choice. See Footnote.

In the simple example, passing a pointer instead of depending on a global array is no particular hardship, but if you have a function with 14 parameters, well I have one thing to say: Do you want to make it work or not?

If you are worried about the overhead of passing 14 parameters, I think that very well may be valid. So---create a struct and populate it with those variables. Define the function to take a pointer to struct and work on it there. That might not seem very straightforward to beginners, but we are beyond that, right?

If you are worried about the amount of work that you have to do to make it happen, well, I can't make that decision for you.

All I have to say is, "Chacun à son goût."

I hate to repeat myself, but I don't claim anything about being the "best" way. It is, however, a way. If you "plan ahead" when writing a sketch, you can implement it in a way that makes it easier to convert to an "applet" function without massive editing changes that risk breaking your carefully crafted and tested sketch code.

Regards,

Dave

In 1972, Edsgar Dijkstra opined:
"It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration."

Dijkstra was trying to drag all of those "seat-of-the-pants" programmers (BASIC, FORTRAN, and others) kicking and screaming into the world of "structured programming." We are still struggling with it.

You pose an interesting question. Here is an explanation and some code that may help. I am interested in feedback. Of course I realize this is not an optimal application, it is designed to be a memory hog .. hence the name. But as far as the idea of encapsulating the code into a class for partitioning purposes.

Sometimes it can be hard to turn some abstract concepts and maybe a lecture into actual code. So I provide some actual code below, so you can see exactly what I am talking about and play with it in practice.

Hope this helps ...

Using a class to encapsulate an application:

The below simple explanation an code will explain how to implement two totally different applications on the arduino with no fear of lingering memory use or conflicting variable/function names.

Naturally both applications need to fit in the flash space provided .. but the memory footprint of each application is partitioned using the class structure.

How it works:
By creating a class and pulling in all the variables and functions, we can run the "loop routine" of the class until it completes (or the current application is changed via an external trigger, etc).

Psudo Code:

loop)

  • see what process to run, a or b.

process a) (in a function)

  • create classA instance and call loop, check reply
  • if reply is true, run it again
  • if reply is false, exit (return to loop)
  • when the function ends, the class and all contained variables are destroyed

process b) (in a function)

  • create classB instance and call loop, check reply
  • if reply is true, run it again
  • if reply is false, exit (return to loop)
  • when the function ends, the class and all contained variables are destroyed
  • do this for as many processes as needed

Actual code.

The code below creates a memory hog application, so big that two of them would blow the RAM limits. Then the application leaves memory and the next one loads and runs. This can run back and forth with no memory leakages.

  • Read the comments and see how you can update a variable and see it use too much ram and crash. In this case the serial readouts will just stop .. application dead :slight_smile:

  • You will want to turn on serial monitor to see the output naturally.

#define APP_NONE 0
#define APP_HOG1 1
#define APP_HOG2 2
#define APP_BREAK_ME 3
#define HOG_FOOD 400

// 3 left out of app count to not break
#define APP_COUNT 2 
#define HOG_DELAY 20

// set to APP_BREAK_ME to see too many hogs created (2 is too many)
// set to APP_HOG1 to see the system create two hogs .. one at a time
//    this method does not blow the RAMs top
int currentApp = APP_HOG1; //APP_BREAK_ME  or APP_HOG1

class Hog1 {
  int _food[HOG_FOOD];
  int _timesFed;
  boolean _isFed;
  int hog1_001;
  int hog1_002;
  int hog1_003;
  int hog1_004;
  
public:
  Hog1(int timesFed) {
    _timesFed = timesFed;
    _isFed = (timesFed > 0);
  };
  Hog1() {
    _timesFed = 0;
    _isFed = false;
  };
  ~Hog1() {
    free(_food);
  };
  boolean loop(){
    _timesFed++;
    if ( _timesFed >= HOG_FOOD ){
      Serial.println("I am a full Hog1");
      return false;
    }
    _food[_timesFed] = 1; // set any value
    Serial.print("Hog1 - fed: ");
    Serial.println(_timesFed, DEC);
    delay(HOG_DELAY);
    return true;
  };
};

class Hog2 {
  int _food[HOG_FOOD];
  int _timesFed;
  boolean _isFed;
  int hog2_001;
  int hog2_002;
  int hog2_003;
  int hog2_004;
  
public:
  Hog2(int timesFed) {
    _timesFed = timesFed;
    _isFed = (timesFed > 0);
  };
  Hog2() {
    _timesFed = 0;
    _isFed = false;
  };
  ~Hog2() {
    free(_food);
  };
  boolean loop(){
    _timesFed++;
    if ( _timesFed >= HOG_FOOD ){
      Serial.println("I am a full Hog2");
      return false;
    }
    _food[_timesFed] = 1000;
    Serial.print("Hog2 - fed: ");
    Serial.println(_timesFed, DEC);
    delay(HOG_DELAY);
    return true;
  };
};

void setup(){
  Serial.begin(9600);
}

void runHog1(){
  Serial.println("Starting runHOG1");
  boolean isRunning = true;
  Hog1 hog = Hog1();
  while (isRunning){
    isRunning = hog.loop();
  }
  Serial.println("Ended runHOG1");
}

void runHog2(){
  Serial.println("Starting runHog2");
  boolean isRunning = true;
  Hog2 hog = Hog2();
  while (isRunning){
    isRunning = hog.loop();
  }
  Serial.println("Ended runHog2");
}

void advanceToNextApp(){
  currentApp++;
  if( currentApp > APP_COUNT )
    currentApp = 1; //or zero or set using external trigger (i.e. serial / button)
 
}

void runTooManyHogs(){
  Serial.println("Starting runTooManyHogs - Feeding Hog1");
  boolean isRunning = true;
  Hog1 hog = Hog1();
  while (isRunning){
    isRunning = hog.loop();
  }
  Serial.println("Done Feeding Hog1, lets feed hog2.");

  isRunning = true;
  Hog2 hog2 = Hog2();
  while (isRunning){
    isRunning = hog2.loop();
  }
  Serial.println("**** Never gets here .. kaboooom");
}

void loop(){
  
  switch (currentApp){
    case APP_NONE : Serial.println("No App");
    break;
    case APP_HOG1 : runHog1();
    break;
    case APP_HOG2 : runHog2();
    break;
    case APP_BREAK_ME : runTooManyHogs();
    break;
  }    
  advanceToNextApp(); 
}