Doing 1000 independent things at the same time with millis()

Here's some code using an array of timers to control 1120 individual WS2182 LEDs with an Arduino Mega

// Use an array of independent millis() timers to track independent events
// Circuit based on https://wokwi.com/projects/287302452979433992
// DaveX 2023-01-31 Apache 2.0

#include <FastLED.h>

#define TIMING 0

#define LED_PIN     3
#define NUM_LEDS    1120
#define NUM_SPARKLE NUM_LEDS // 800
#define LED_TYPE    WS2812
#define COLOR_ORDER GRB
#define MAX_OFF_MS 60000L

CRGB leds[NUM_LEDS];

#define NUM_RINGS (sizeof(led_count) / sizeof(led_count[0]))
#define FIRE_WIDTH 64
#define FIRE_HEIGHT NUM_RINGS

unsigned long next_change_time[NUM_LEDS]; // memory for timers

void setup() {
  Serial.begin(115200);
  FastLED.addLeds<LED_TYPE, LED_PIN, COLOR_ORDER>(leds, NUM_LEDS);
  for (int i = 0 ; i < NUM_LEDS; i++) {
    leds[i] = CRGB::Black;
    next_change_time[i] = random(MAX_OFF_MS);
  }
}

void loop() {
  unsigned long t1 = millis();
  int changes = 0;
  CRGB *led = leds;

  for (int i = 0; i < NUM_SPARKLE; ) {
    if ((signed long)(t1 - next_change_time[i]) > 0) {
      // time for a change
      if (leds[i]) { // lit?
        leds[i] = CRGB::Black;
        next_change_time[i] = t1 + random(500, MAX_OFF_MS);
      } else {
        // https://github.com/FastLED/FastLED/wiki/Pixel-reference
        leds[i] = CHSV(random(256), 255, 255);
        next_change_time[i] = t1 + 1000;
      }
      changes++;
    }
    i++;
  }
  if (changes) {
    FastLED.show();
  //  Serial.print('*');
  } else {
   // Serial.print('.');
  }

}

It uses an array to keep track of a separate timer for each LED, and the FastLED data to keep track of the state of each LED.

This code uses about all of the RAM in the Mega, and with the MAX_OFF_MS=60000, loop barely keeps up with writing all the changes to the LEDs.

It was derived from the simulation circuit at rings1629_Sol.ino - Wokwi Arduino and ESP32 Simulator and the ideas in and others Demonstration code for several things at the same time

2 Likes

taking out the "(signed long)" speeds things up.

That would make it an unsigned long and nearly always be true.

I was trying to schedule future events so I didn't need two unsigned longs per pixel, (for last and interval) just the time of the future event. Which when differenced with millis() is either negative while it is still in the future, or positive once the time has passed.

loop()-speed wise, theres a few other tricks to try -- you can rate-limit the expensive schedule checking and its LED updating, which groups the updates, or you can make the off-interval longer so there are less updated pixels per loop(). I like the rate-limiting scheme:

// Use an array of independent millis() timers to track independent events
// Circuit based on https://wokwi.com/projects/287302452979433992
// for https://forum.arduino.cc/t/doing-1000-independent-things-at-the-same-time-with-millis/1084860/
// DaveX 2023-01-31 Apache 2.0

#include <FastLED.h>

#define TIMING 0

#define LED_PIN     3
#define NUM_LEDS    1000 //1120
#define NUM_SPARKLE NUM_LEDS // 800
#define LED_TYPE    WS2812
#define COLOR_ORDER GRB
#define MAX_OFF_MS 5000L


CRGB leds[NUM_LEDS];

#define NUM_RINGS (sizeof(led_count) / sizeof(led_count[0]))
#define FIRE_WIDTH 64
#define FIRE_HEIGHT NUM_RINGS

const int debug = 0;
unsigned long next_change_time[NUM_LEDS]; // memory for timers

void setup() {
  Serial.begin(115200);
  FastLED.addLeds<LED_TYPE, LED_PIN, COLOR_ORDER>(leds, NUM_LEDS);
  for (int i = 0 ; i < NUM_LEDS; i++) {
    leds[i] = CRGB::Black;
    next_change_time[i] = random(MAX_OFF_MS);
  }
}

void loop() {
  unsigned long t1 = millis();
  int changes = 0;
  CRGB *led = leds;
  static unsigned long last = 0;
  const unsigned long interval = 50 ; //ms

  if ( t1 - last >= interval) { // limit rate
    last += interval;
    for (int i = 0; i < NUM_SPARKLE; ) {
      if ((signed long)(t1 - next_change_time[i]) > 0) {
        // time for a change
        if (leds[i]) { // lit?
          leds[i] = CRGB::Black;
          next_change_time[i] += random(500, MAX_OFF_MS);
        } else { // light it up
          // https://github.com/FastLED/FastLED/wiki/Pixel-reference
          leds[i] = CHSV(random(256), 255, 255);
          next_change_time[i] += 1000;
        }
        changes++;
      }
      i++;
    }
  }
  if (changes) { // update LEDS
    FastLED.show();
    if(debug){Serial.print('*');Serial.print(changes);}
  } else {
    if(debug){Serial.print('.');}
  }
}

that won't work if you keep this animation running long enough

basically you want to check if t1 > next_change_time[i] but that won't work at rollover time if you keep the animation going for 50 days

Given MAX_OFF_MS is less that 65535, you could use an uint16_t to store the next_change_time and save half the memory if you were going for start time and ∆t to be rollover safe.

(also there are many pieces unused in your code that you could clean up to keep the demo simple)

may be something like this would do

#include <FastLED.h>

#define LED_PIN     3
#define NUM_LEDS    1120
#define LED_TYPE    WS2812
#define COLOR_ORDER GRB
#define MAX_OFF_MS 60000L

CRGB leds[NUM_LEDS];
uint16_t changeInDeltaT[NUM_LEDS]; // memory for timers

void setup() {
  Serial.begin(115200);
  FastLED.addLeds<LED_TYPE, LED_PIN, COLOR_ORDER>(leds, NUM_LEDS);
  for (int i = 0 ; i < NUM_LEDS; i++) {
    leds[i] = CRGB::Black;
    changeInDeltaT[i] = random(MAX_OFF_MS);
  }
}

void loop() {
  static uint32_t oldMillis = millis();
  uint32_t currentMillis = millis();
  uint32_t deltaTLoop = currentMillis - oldMillis;
  bool stripNeedsUpdate = false;

  for (uint16_t i = 0; i < NUM_LEDS; i++) {
    if (changeInDeltaT[i] == 0) { // time to change
      stripNeedsUpdate = true;
      changeInDeltaT[i] =  random(MAX_OFF_MS);
      if (leds[i] == CRGB(CRGB::Black))
        leds[i] = CHSV(random(256), 255, 255);
      else
        leds[i] = CRGB::Black;
    } else {
      if (deltaTLoop > changeInDeltaT[i])
        changeInDeltaT[i] = 0;
      else
        changeInDeltaT[i] -= deltaTLoop;
    }
  }
  if (stripNeedsUpdate) FastLED.show();
  oldMillis = currentMillis;
}

71% of the memory used instead of 88%

No, this math protects against that sort of rollover:

The differencing makes it an interval mod ULONG_MAX, and the cast offsets the range to both sides of zero so you can use it as an indicator variable for past and future. (The " What if I really need to compare timestamps" section of the excellent programming - How can I handle the millis() rollover? - Arduino Stack Exchange explains this.)

However this math does limit the next_change_time[i] to ~25 days (half of ULONG_MAX or just LONG_MAX) because longer intervals into the future would rollover into the past.

Interesting. I wanted to use the full set of 1630 LEDs from the source simulation, but could only get to 1120 or so before things started to crash.

If I switch the my code to uint16_t get the same:

Sketch uses 5666 bytes (2%) of program storage space. Maximum is 253952 bytes.
Global variables use 5880 bytes (71%) of dynamic memory, leaving 2312 bytes for local variables. Maximum is 8192 bytes.

With my code switched to uint16_t it can get to 1550 LEDs at 98% RAM:


// Use an array of independent millis() timers to track independent events
// Circuit based on https://wokwi.com/projects/287302452979433992
// for https://forum.arduino.cc/t/doing-1000-independent-things-at-the-same-time-with-millis/1084860/
// DaveX 2023-01-31 Apache 2.0

// uses millis() per https://www.gammon.com.au/millis 
// and comparison of future timestamps per https://arduino.stackexchange.com/a/12588/6628

#include <FastLED.h>

#define TIMING 0

#define LED_PIN     3
#define NUM_LEDS    1550 //1120 max before memory issues //1630 LEDs total
#define NUM_SPARKLE NUM_LEDS // 800
#define LED_TYPE    WS2812
#define COLOR_ORDER GRB
#define MAX_OFF_MS 5000L


CRGB leds[NUM_LEDS];

#define NUM_RINGS (sizeof(led_count) / sizeof(led_count[0]))
#define FIRE_WIDTH 64
#define FIRE_HEIGHT NUM_RINGS

const int debug = 0;
uint16_t next_change_time[NUM_LEDS]; // memory for timers

void setup() {
  Serial.begin(115200);
  FastLED.addLeds<LED_TYPE, LED_PIN, COLOR_ORDER>(leds, NUM_LEDS);
  for (int i = 0 ; i < NUM_LEDS; i++) {
    leds[i] = CRGB::Black;
    next_change_time[i] = random(MAX_OFF_MS);
  }
}

void loop() {
  unsigned long t1 = millis();
  int changes = 0;
  CRGB *led = leds;
  static unsigned long last = 0;
  const unsigned long interval = 50 ; //ms

  if ( t1 - last >= interval) { // limit rate
    last += interval;
    for (int i = 0; i < NUM_SPARKLE; ) {
      if ((signed long)(t1 - next_change_time[i]) > 0) {
        // time for a change
        if (leds[i]) { // lit?
          leds[i] = CRGB::Black;
          next_change_time[i] += random(500, MAX_OFF_MS);
        } else { // light it up
          // https://github.com/FastLED/FastLED/wiki/Pixel-reference
          leds[i] = CHSV(random(256), 255, 255);
          next_change_time[i] += 1000;
        }
        changes++;
      }
      i++;
    }
  }
  if (changes) { // update LEDS
    FastLED.show();
    if(debug){Serial.print('*');Serial.print(changes);}
  } else {
    if(debug){Serial.print('.');}
  }
}

I'm familiar with unsigned math.

here it won't work because the math (t1 - next_change_time[i]) is conducted as unsigned long (indeed modulo ULONG_MAX) but the result is always positive or null as both are unsigned. so even if you cast to signed later on, you will still get a positive answer. Your if will only catch == 0

EDIT: does go negative if the delta is has a 1 in the MSb (is > ULONG_MAX /2 )

So this has no chance of printing "negative"?

// https://wokwi.com/projects/355481131343303681 
// for https://forum.arduino.cc/t/doing-1000-independent-things-at-the-same-time-with-millis/1084860
#include <limits.h>

void setup() {
  // put your setup code here, to run once:

  Serial.begin(115200);

  unsigned long t1  = ULONG_MAX - 100UL;
  unsigned long t2  = ULONG_MAX - 10UL;

      if ((signed long)(t1 - t2) > 0) {
        Serial.println("positive");
      }else{
        Serial.println("negative");
      }
}

void loop() {
  // put your main code here, to run repeatedly:

}




// prints "negative"

right indeed, yes of course it does if you go above ULONG_MAX / 2

is it how you intended to use it? (in your example t1 is not > t2 so you should not trigger)

EDIT: I missed that you had caught that indeed

but it's true with 16 bits or 32 bits, isn't it?

If what goes above ULONG_MAX/2?

if the unsigned difference is above ULONG_MAX / 2

but you are right, it should work

actually something is bothering me mathematically but I could not pin point it

Now I think I do:

say
current millis is t1 = ULONG_MAX / 2
change time is t2 = 0 ( or more, rolled over)

You are pretty far from needing to change the led but the ∆t will say positive (so trigger the led change in your formula)

      if ((signed long)(t1 - t2) > 0) {
        Serial.println("positive");
      }else{
        Serial.println("negative");
      }

But that's theoreticalI guess as it can't happen with you values you are using to calculate the change moment, it's never that far in the future

The unsigned difference could get be above ULONG_MAX/2 if you set intervals larger than 25 days or maybe if you have a loop time larger than 25 days. Otherwise it is rollover-safe because the modulo math with unsigned differences takes care of itself.

I like scheduling future events because they require less storage--only the next time, rather than both the last time and the interval between changes.

Sometimes translating the time of future events into the last + interval paradigm is awkward. If you have just one LED or oven, or relay, keeping track of the last time you changed it is reasonable, but if you have many, it isn't as clear.

///

Looking at your next post, the normal rollover problem solution can have the same theoretical problem if the check time or intervals are somehow larger than ULONG_MAX.

If you have control over your looping or testing time, you could avoid the cast and use a test like:

      const unsigned max_check_interval = 10000UL; // <10s sleeps
      ...
      if (t1 - t2 + max_check_interval > max_check_interval) {
        Serial.println("positive");
      }else{
        Serial.println("negative");
      }

...and get proper behavior for delta-ts of up to nearly the full range. (max_check_interval = ULONG_MAX/2; would make it equivalent to the cast) But the cast seems easier to explain as "treat the difference between the two timestamps as a positive or negative difference"

Indeed

Per discussion elsewhere, I noticed that if you:

  • #include <time.h>
  • assume millis() and micros() each are the same as time_t (which happens to be the case (true on Unos ))

Then you can simplify the code to avoid the (signed long)(now - next) casting and use difftime(now,next) thus:

// Use an array of independent millis() timers to track independent events
// Circuit based on https://wokwi.com/projects/287302452979433992
// for https://forum.arduino.cc/t/doing-1000-independent-things-at-the-same-time-with-millis/1084860/
// DaveX 2023-01-31 Apache 2.0

// uses millis() per https://www.gammon.com.au/millis 
// and comparison of future timestamps per https://arduino.stackexchange.com/a/12588/6628
// or 
// 2023-12-07 -- modified to to abuse time.h, time_t and difftime()


#include <time.h> // https://www.nongnu.org/avr-libc/user-manual/modules.html
#include <FastLED.h> // https://github.com/FastLED/FastLED

#define TIMING 0

#define LED_PIN     3
#define NUM_LEDS    1000 //1120
#define NUM_SPARKLE NUM_LEDS // 800
#define LED_TYPE    WS2812
#define COLOR_ORDER GRB
#define MIN_OFF_MS 5000L
#define MAX_OFF_MS 10000L
#define MIN_ON_MS 1000L
#define MAX_ON_MS 1000L


CRGB leds[NUM_LEDS];

#define NUM_RINGS (sizeof(led_count) / sizeof(led_count[0]))
#define FIRE_WIDTH 64
#define FIRE_HEIGHT NUM_RINGS

const int debug = 0;
time_t next_change_time[NUM_LEDS]; // memory for timers

void setup() {
  Serial.begin(115200);
  FastLED.addLeds<LED_TYPE, LED_PIN, COLOR_ORDER>(leds, NUM_LEDS);
  for (int i = 0 ; i < NUM_LEDS; i++) {
    leds[i] = CRGB::Black;
    next_change_time[i] = random(MAX_OFF_MS);
  }
  Serial.print("LEDs On:{");
  Serial.print(MIN_ON_MS);
  Serial.print(',');
  Serial.print(MAX_ON_MS);
  Serial.print("}ms Off: {");
  Serial.print(MIN_OFF_MS);
  Serial.print(',');
  Serial.print(MAX_OFF_MS);
  Serial.println("}ms");
}

void loop() {
  unsigned long t1 = millis();
  int changes = 0;
  CRGB *led = leds;
  static unsigned long last = 0;
  const unsigned long interval = 50 ; //ms

  if ( t1 - last >= interval) { // limit rate
    last += interval;
    for (int i = 0; i < NUM_SPARKLE; ) {
//      if ((signed long)(t1 - next_change_time[i]) > 0) {
      if (difftime(t1, next_change_time[i]) > 0) { // future event slipped into the past
        // time for a change
        if (leds[i]) { // lit?
          leds[i] = CRGB::Black;
          next_change_time[i] += random(MIN_OFF_MS, MAX_OFF_MS);
        } else { // light it up
          // https://github.com/FastLED/FastLED/wiki/Pixel-reference
          leds[i] = CHSV(random(256), 255, 255);
          next_change_time[i] += random(MIN_ON_MS, MAX_ON_MS);;
        }
        changes++;
      }
      i++;
    }
  }
  if (changes) { // update LEDS
    FastLED.show();
    if(debug){Serial.print('*');Serial.print(changes);}
  } else {
    if(debug){Serial.print('.');}
  }
  report();
}

void report(){
  const int interval = 1000;
  static time_t last = -interval;
  static long count = 0;
  ++count;
  if(difftime(millis(), last) < interval ) return;
  last += interval;
  Serial.print("loop() count:");
  Serial.print(count);
  Serial.print("/sec or ms:");
  Serial.print(1000.0/count);
  Serial.println();
  count=0;
}

where ?

(I just posted an edit to the code, because it was based on an earlier version which had a memory problem)

I had a cast in the time comparison to make the if((signed long) (now - next) >0) test for events in the future versus the past:

if now and next are uint32_t then the subtraction is carried out as unsigned, only afterwards it's promoted to a signed long in an undocumented way (as unsigned fit larger values than signed)...

what are you really trying to test? if (now > next) ??

Yes. If you try to test if(now - last[i] >= interval[i] with arrays of last[i] and interval[i] you need double the storage required for if(difftime(now,nextEvent[i]) > 0 )

if you expect short intervals, the interval array's type could be uint8_t or uint16_t - it does not need to be uint32_t

if you compare now to next directly, you don't handle rollover properly

I suppose I'm assuming difftime() would handle rollover correctly, or at least as well as if((signed long) (now - next) >= 0) Test included below.

An issue I have with the standard if(now - last >= interval) scheme is trying to schedule a "feed my chickens every 6 hours starting now"-type use-case. It is easy to gloss over the ability of the the standard scheme to handle handle this, since BWoD example starts so quickly. The standard scheme can work for this use-case if you pre-rollover last to be -interval (or update interval from 0 to 6hrs after the first event) but it is awkward to explain. Another workaround is to duplicate the task in setup(), which is easy to explain, but is more awkward to code. Using difftime() or (signed long)(now - next) seems a bit more clear. Especially if you are dealing with multiple events of differing intervals.

#include <time.h>

extern unsigned long timer0_millis;

void setup() {
  Serial.begin(115200);
  // force millis rollover soon:
  noInterrupts();
  timer0_millis = 0UL - 5000;
  interrupts();
}

void loop() {
  // put your main code here, to run repeatedly:
  every6HrStartingNow();
  every6HrStartingNowV2();
  report();
}

void every6HrStartingNow(void) {
  const unsigned long interval = 1000UL * 3600 * 6;
  static unsigned long last = -interval;
  unsigned long now = millis();
  if ( now - last >= interval) {
    last = now;
    Serial.print("every6HrStartingNow  standard: "); Serial.println(now);
  }
}

void every6HrStartingNowV2(void) {
  const unsigned long interval = 1000UL * 3600 * 6;
  static time_t next = 0;
  unsigned long now = millis();
  if ( difftime(now, next) >= 0 ) {
    next = now + interval;
    Serial.print("every6HrStartingNowV2 difftime: "); Serial.println(now);
  }
}


void report(void) {
  const unsigned long interval = 1000;
  static time_t next = millis();
  unsigned long now = millis();
  if ( difftime(now, next) >= 0 ) {
    next = now + interval;
    Serial.print("millis(): "); Serial.println(now);
  }
}

gives:

millis(): 4294962296
millis(): 4294963296
millis(): 4294964296
millis(): 4294965296
millis(): 4294966296
millis(): 0
every6HrStartingNow  standard: 0
every6HrStartingNowV2 difftime: 0
millis(): 1000
millis(): 2000
millis(): 3000
millis(): 4000
millis(): 5000
millis(): 6000
millis(): 7000
millis(): 8000
millis(): 9000
millis(): 10000