var[i] = millis() * 100, will I run out of memory?

I'm working on a sketch that buffers incoming midi keyup events and executes a non-blocking led animation after those events have registered.
To continue with the path I'm on, though, I have to keep an array of ~88 booleans and ~100 instances of var = millis(), a 32 bit value, in memory. The number is this high to account for the volume of expected keyup events over N amount of time.
After this many had been set, the array would rollover, so that would be the maximum amount of values stored, but I'm thinking this is a fools errand. Is this way too many values to attempt to store?

I'm using a Leonardo right now but plan to transition to an ESP32.

100 X 32bit = 3.2kb + 0.88kb = 4.08KB + all other running tasks.

Leonardo has 32kb flash memory and 2.5kb sram

ESP32 has 448kb ROM 520KB 16KB SRAM

I've just never pushed a microcontroller this hard and am wondering if this is an okay thing to do.

I have used array sizes on the ESP32, numbering in the 300 to 500 range; including floats, without issue.

On one project using a ESP32WROVER, I make a ring buffer sized to use the entirety of a 4MB PSRAM bank.


Whiles the ESP32 does not have a millis() counter, it does have a micros counter that rolls over after 200+ years. If you need to count the time and the millis() rolls over you may not produce the correct time lapsed value. OK, well you can but don't forget to write millis() a rollover code thingy to compensate for the rollover.

As far as wanting an array to roll over., it would be best to use a ringbuffer instead of an array. The ESP32 API has a freeRTOS ringbuffer that can be set to do house keeping chores, like rolling over; see FreeRTOS (Supplemental Features) - ESP32 - — ESP-IDF Programming Guide latest documentation

Idahowalker:
I have used array sizes on the ESP32, numbering in the 300 to 500 range; including floats, without issue.

On one project using a ESP32WROVER, I make a ring buffer sized to use the entirety of a 4MB PSRAM bank.


Whiles the ESP32 does not have a millis() counter, it does have a micros counter that rolls over after 200+ years. If you need to count the time and the millis() rolls over you may not produce the correct time lapsed value. OK, well you can but don't forget to write millis() a rollover code thingy to compensate for the rollover.

As far as wanting an array to roll over., it would be best to use a ringbuffer instead of an array. The ESP32 API has a freeRTOS ringbuffer that can be set to do house keeping chores, like rolling over; see FreeRTOS (Supplemental Features) - ESP32 - — ESP-IDF Programming Guide latest documentation

Awesome. Thank you very much.

Whiles the ESP32 does not have a millis() counter, it does have a micros counter that rolls over after 200+ years. If you need to count the time and the millis() rolls over you may not produce the correct time lapsed value.

Doesn't the Arduino ESP32 core support millis() in the usual way? Rollover shouldn't be a problem with correct code.

I don't think your calculation of the memory used is correct. 32bits is 4 bytes, a boolean 1 byte. 5 x 88 = 440 bytes, or 100x4 +88 = 488 bytes.
You have cobnfused kb and KB, b for bits abd B for Bytes.

440 bytes will fit on an Uno no problem.

countrypaul:
I don't think your calculation of the memory used is correct. 32bits is 4 bytes, a boolean 1 byte. 5 x 88 = 440 bytes, or 100x4 +88 = 448 bytes.
You have cobnfused kb and KB, b for bits abd B for Bytes.

440 bytes will fit on an Uno no problem.

Aw geez, you are correct. I was treating bits as bytes. Thank you.

aarg:
Doesn't the Arduino ESP32 core support millis() in the usual way? Rollover shouldn't be a problem with correct code.

I've not looked to deeply into the Arduino Core millis() when the ESP32 has its own micro based tick counter that takes 200+ years to roll over.

I, as much as possible, avoid using the ESP32 Arduino Core. I have found that by directly using the ESP32 API, things run faster, better and I gain more control over the process. The Arduino ESP32 Core, a wrapper around the ESP32 API, is not always the most efficient way of doing the thing on an ESP32.

Yup, code should be written to handle millis() roll over, was just alerting the fellow to such a thing.

Whiles the ESP32 does not have a millis() counter

What exactly do you mean by that ?

it does have a micros counter that rolls over after 200+ years

How many microseconds there are in 200 years and what type of variable is the value returned in ?

UKHeliBob:
What exactly do you mean by that ?
How many microseconds there are in 200 years and what type of variable is the value returned in ?

calculator.com https://www.calculator.com/, uint64_t.

ESP32 API API Reference - ESP32 - — ESP-IDF Programming Guide latest documentation

I once went looking for the ESP32 timer rollover, found it, remembered that is was over 200 years, and went on my way.


How many uS in 200 years = 6.3072E+15 = 6,311,390,400,000,000
Size of 2^64 =1.845E+19 = 18,446,744,073,709,551,616

Idahowalker:
calculator.com https://www.calculator.com/, uint64_t.

ESP32 API API Reference - ESP32 - — ESP-IDF Programming Guide latest documentation

I once went looking for the ESP32 timer rollover, found it, remembered that is was over 200 years, and went on my way.


How many uS in 200 years = 6.3072E+15 = 6,311,390,400,000,000
Size of uint64_t = 2^64 = 6.3072E+15 = 18,446,744,073,709,551,616

That looks wrong - should be 2^64 =1.845E+19 = 18,446,744,073,709,551,616

Can you change the millis() underlying counter variable to uint64_t? My project needs to keep track of the age of the universe...

countrypaul:
That looks wrong - should be 2^64 =1.845E+19 = 18,446,744,073,709,551,616

Oi, yes I pasted the wrong number thanks I'll correct it; +1 to you.

Idahowalker:
I, as much as possible, avoid using the ESP32 Arduino Core.

What, exactly, does that mean? You instead use the Espressif ESP-IDF? Seems to me that in doing that you would lose access to the thousands of libraries that are targeted to the Arduino Ecosystem. Even though many of them are not well-written, others actually save you the trouble of reinventing the wheel when it comes to interfacing sensors, displays, etc.

gfvalvo:
What, exactly, does that mean? You instead use the Espressif ESP-IDF? Seems to me that in doing that you would lose access to the thousands of libraries that are targeted to the Arduino Ecosystem.

What it means:
This was written in the Arduino IDE bypassing the ESP32 core:

#include <driver/adc.h>
#include "sdkconfig.h"
#include "esp_system.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
////
#define evtDoParticleRead   ( 1 << 0 ) // declare an event
EventGroupHandle_t eg; // variable for the event group handle
////
esp_timer_handle_t oneshot_timer; //veriable to store the hardware timer handle
//
void IRAM_ATTR oneshot_timer_callback( void* arg )
{
  BaseType_t xHigherPriorityTaskWoken;
  xEventGroupSetBitsFromISR( eg, evtDoParticleRead, &xHigherPriorityTaskWoken ); //freeRTOS event trigger made for ISR's
}
////
void setup()
{
  eg = xEventGroupCreate(); // get an event group handle
  //
  gpio_config_t io_cfg = {}; // initialize the gpio configuration structure
  io_cfg.mode = GPIO_MODE_OUTPUT; // set gpio mode
  //bit mask of the pins to set
  io_cfg.pin_bit_mask = ( (1ULL << GPIO_NUM_4) ); // assign gpio number to be configured
  //configure GPIO with the given settings
  gpio_config(&io_cfg); // configure the gpio based upon the parameters as set in the configuration structure
  gpio_set_level( GPIO_NUM_4, LOW); // set air particle sensor trigger pin to LOW
  // set up A:D channels
  // https://dl.espressif.com/doc/esp-idf/latest/api-reference/peripherals/adc.html
  adc1_config_width(ADC_WIDTH_12Bit);
  adc1_config_channel_atten(ADC1_CHANNEL_4, ADC_ATTEN_DB_11);// using GPIO 32
  //
  // https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/esp_timer.html?highlight=hardware%20timer High Resoultion Timer API
  esp_timer_create_args_t oneshot_timer_args = {}; // initialize High Resoulition Timer (HRT) configuration structure
  oneshot_timer_args.callback = &oneshot_timer_callback; // configure for callback, name of callback function
  esp_timer_create( &oneshot_timer_args, &oneshot_timer ); // assign configuration to the HRT, receive timer handle
  //
  xTaskCreatePinnedToCore( fDoParticleDetector, "fDoParticleDetector", 5000, NULL, 3, NULL, 1 ); // assign all to core 1
} // end void setup()
////
void fDoParticleDetector( void * parameter )
{
  float ADbits = 4095.0f;
  float uPvolts = 3.3f;
  float adcvalue = 0.0f;
  for (;;)
  {
    //enable sensor led
    gpio_set_level( GPIO_NUM_4, HIGH ); // set gpio 4 to high
    esp_timer_start_once( oneshot_timer, 280 ); // trigger one shot timer for a 280uS timeout
    //esp_timer_start_once( oneshot_timer, 380 ); //
    xEventGroupWaitBits (eg, evtDoParticleRead, pdTRUE, pdTRUE, portMAX_DELAY ); // event will be triggered by the timer expiring, wait here for the 280uS
    //take a raw ADC reading from the dust sensor
    adcvalue = float( adc1_get_raw(ADC1_CHANNEL_4) );
    //Shut off the sensor LED
    gpio_set_level( GPIO_NUM_4, LOW );
    //calculate voltage
   adcvalue = ( uPvolts * adcvalue ) / ADbits; // calculate sensor voltage
    log_i( "%f", adcvalue ); // print the ADC value
    vTaskDelay( 125 );
  }
  vTaskDelete( NULL );
}// end fDoParticleDetector()
////
void loop() {}

The code runs faster, it does not use the overhead of the Arduino ES32 IDE core.
The code for the A:D converter gives a more accurate result then if I used the Arduino ESP32 IDE core.
The code is, when added to the larger project, not blocking, even though there is a 280uS delay

There are some ESP32 features that are not in the Arduino core.

IDF was not used to write that code. All done in the Arduino IDE.

Retains my access to

the thousands of libraries that are targeted to the Arduino Ecosystem.

Ahh... so you deliberately eschew the conveniences provided the Arduino core ... things like digitalRead(), digitalWrite(), analogRead(), etc and use more efficient ESP-specific constructs.

Of course, once you include a library written for a generic "Arduino" board, that efficiency goes out the window.

countrypaul:
That looks wrong - should be 2^64 =1.845E+19 = 18,446,744,073,709,551,616

For reasons I can't fathom, it looks like some of the API functions used a SIGNED 64-bit number:


So, the maximum value would be 2^63-1.

gfvalvo:
Ahh... so you deliberately eschew the conveniences provided the Arduino core ... things like digitalRead(), digitalWrite(), analogRead(), etc and use more efficient ESP-specific constructs.

Of course, once you include a library written for a generic "Arduino" board, that efficiency goes out the window.

Correct.

I've rewritten several libraries using SPI to gain the ESP32 SPi features. I can use a MPU9250 at 7Mhz with background receiving and sending.

gfvalvo:
For reasons I can't fathom, it looks like some of the API functions used a SIGNED 64-bit number:


So, the maximum value would be 2^63-1.

Could be that the author never tested the limit, after all which of us would wait that long for the test to finish? :smiley:

One of the most common problems on the forum seems to be unsigned/signed confusion and size of the different datatypes, something I'm sure we all do from time to time.

I found a more elegant method than what I had planned.
Using struct I was able to make a custom data array of all 88 keys, that can hold all the event data I need, conveniently indexed to key number.

typedef struct{
  uint8_t      keyStruck;
  uint8_t      channel;
  uint8_t      velocity;
  uint8_t      keyLight[1];
  bool     isDown;
  bool     recentlyReleased;
  uint32_t lastReleased;
}keyState;

keyState keyBuffer[88];

I haven't had performance issues, but I haven't pushed it too far yet. Thank you all for your help.