Memory optimization

I'm working on a model railroad layout. It's quite big and will have many houses, street lights, and other lights. The total number is hard to predict at this stage, but around thousand is a good guess to start with. I'm talking about individually controlled "channels", not actual light, since there will be multiple LEDs on each "channel".
The layout has a computer controlled day/night cycle and has a proprietary device that communicates with the computer and turns on and off various lights. The device has 40 outputs and is stackable, but quite expensive. The decision we made is to see if we can come up with our own solution.

Here comes the Arduino. I picked up a couple of 328 chips which I plan to turn into a bunch of standalones, each for controlling a part of model layout, and each triggered by the proprietary device when the night comes.
I'm using shift registers and can turn on and off individual lights as I please. Light effects that I am interested in implementing include, for example a broken neon light, regular neon light (the turning on effect), house room light (randomly turning on and off during the night), streetlight (fade on), fire, TV etc...
Blink without delay concept proved to be a suitable method for controlling individual lights, but this is where I fear I am about to hit the wall. The way my code works is to keep track of time (millis) when each output last changed state, the state itself, various timings for each effect (how long is the output high or low), etc. All this is stored in various arrays.
I'm afraid that by using longs to keep track of time for each individual output I will quickly run out of memory. ATM I'd be happy if I could run 64 outputs with various light effects, but my preliminary calculations show that it will be a close call to fit it in 2KB available, and I'd be happier if I could run 128 outputs.

The first version of the code I wrote was rather generous when using memory (for each effect I made an array large enough for all the outputs regardless of all those outputs being used by that effect or not. I realized my sin and am in the process of rewriting it so that arrays are automatically made just big enough and no larger than required. I am experiencing a bug there I am yet to figure out, but that's not the point.

I came here to ask you if you have any general suggestions on how to reduce the memory requirements even further. I don't expect you to rewrite my code, just to point me in the direction of a good memory optimization techniques (if such exist). I understand that 328 may be too small for this and am prepared for the possibility that I'll need a bigger chip, but I'd like to explore my options with 328, which I have at hand.

well the 1284p is a bigger chip weighing in at 16kb of ram compared to the 2 on the 328p, but before we go there, you should post (or attach) your code

while we may not re-write it, we need to be able to see what you have, otherwise we are throwing darts in an ocean while hoping to hit a mermaid

The problem you've described cries out for an object oriented solution. Each channel would have an extra two bytes for the VMT pointer but you would be able to create classes of channels that only used the memory necessary to manage that channel.

I'm afraid that by using longs to keep track of time for each individual output I will quickly run out of memory.

Then don't. I frequently use unsigned short. As long as everything in the expression is the same size the math still works...

static unsigned short Previous;
const unsigned short Rate = 1313;

void loop( void )
{
  unsigned short Current;

  Current = millis();

  if ( Current - Previous >= Rate )  // If necessary, use type casting here to ensure everything is unsigned short.
  {
    // Do your thing
    Previous += Rate;
  }
}

Don't use unsigned char unless you are very very careful not to overrun the 255 millisecond window (which is easy to do).

You're right. I was going to attach them and forgot :slight_smile:
Here they are.
"Original" is the first incarnation, and is fairly obsolete, but might be simpler to understand.
"Version2" is newer and still in the making code, suffers from a few bugs where the last output on the first shift registers is constantly high, and if neonFlicker effect does not start from the first pin, I get weird results (blinking outputs, but in much slower way than supposed to). I haven't had the time to track down these bugs and squish them, yet.
V2 is the one where I attempt to optimize array sizes.

Original.txt (8.41 KB)

Version2.txt (8.79 KB)

255 milisecond?
I need the layout to work for around 10 hours nonstop, with day/nigh cycle lasting about 20 minutes. I could reset the chip each "evening" to start the timers from scratch, but I still need it to count for 10 minutes without interrupting it.

just at a glance one thing that will help a little is ditching int

if your values never go above 255 use byte ... just in your globals you can save 14 bytes with a simple type change which is almost 2 longs, same for your for loops, which comes and goes as functions are called, but if your running tight ...

course how can you tell if your running tight? A not perfect way but a good idea for debugging is

int freeRam () {
  extern int __heap_start, *__brkval; 
  int v; 
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval); 
}

then just place a Serial.println(freeRam()) wherever you want to check (course this uses a few bytes of ram itself)

How about abandoning the arrays you have in favour of one that just holds the state of each light. A byte would do it, even better, use a bit. When it becomes time to turn lights on, use your random function to pick a light on each iteration around loop and put it through the flicker routine. Gradually, all the lights in town will come on.

If the random choice picks a light that's already in the desired state, iterate through the array until you find one that isn't, if any.

For extra realism, pick say ten lights at a time and set up the millis arrays just to hold their timers and put them through the flicker routine, each with slightly different timings.

With this model, you should be able to do thousands of lights with a single 328 (plus a boatload of shift registers!)

AVR035 app. note may be helpful.

Thanks guys.
Bill, I'd like the lights to turn on and off randomly during the night, however, without keeping track of time of the last change a random function will at some point cause the light to turn off and back on immediately and vice versa, which might not be such a bad thing anyway (I'm talking to myself at this point, don't mind me). After all it wouldn't be a first time for me to turn off the lights only to realize I forgot something and turn it on again.
I suppose if I pick a random number between 0 and a lot, I could only change state if the number is smaller than not-a-big-number if I want generally slow cycles, and bigger than not-so big-number if I want fast cycles. Juggling those three numbers might just as well prove to provide satisfactory results at no memory cost other than output state.
I'll have to try this.

Thanks man. I knew many folks + me are smarter than just me.
Magician, I'm bookmarking that!