SRAM Limitations When Creating Data Arrays

Hello,

I am pretty new to C++ and Arduino. I am attempting to record force from an fsr sensor and timestamps for each point of data. I want to be able to record a large amount of this data, say a million points and save it to an sd card so that I can analyze it later.
The way I have the code setup right now I am running into SRAM capacity limitations. Any recommendations would be appreciated on how I can fix this, if there are other ways of handling this data I am open to changing the way I have to sketch setup. It is important that I keep the write speed high, between 10ms and 1ms ideally. The system needs to standalone, aka it will not be connected to a PC while recording this data.

Hardware:
Arduino Uno r3

int fsrPin = 0;     // the FSR and 10K pulldown are connected to a0
int fsrReading;     // the analog reading from the FSR resistor divider
int fsrVoltage;     // the analog reading converted to voltage
unsigned long fsrResistance;  // The voltage converted to resistance, can be very big so make "long"
unsigned long fsrConductance;
long fsrForce;       // Finally, the resistance converted to force
const int MAX_READINGS = 10000; // Maximum number of readings
int TAKE_READINGS = 10; // First time readings are stored
long readings[2][MAX_READINGS]; // Stores timestamps and forces
int j = 0; // Index for the next reading
unsigned long timestamp; // Get current time in milliseconds

const int BUTTON_PIN = 8; // Button pin
bool recording = true; // State to track if we are recording
bool lastButtonState = HIGH; // Assume button starts in not-pressed state (using internal pull-up) 
bool pressed = false;
#include <SD.h>

void setup(void) {
  Serial.begin(115200);   // We'll send debugging information via the Serial monitor
// setup pin modes 
  pinMode (BUTTON_PIN, INPUT_PULLUP);
}

void writeToSD(int j, long readings[2][MAX_READINGS]) {
  File dataFile = SD.open("results_" + String(j) + ".txt", FILE_WRITE);
  if (dataFile) {
    int i = j - TAKE_READINGS;
    if (i >= j){
      // dataFile.print("Timestamp:");
      dataFile.print(readings[0][i]);
      dataFile.println(", ");
      // dataFile.print("FSR Force:");
      dataFile.println(readings[1][i]);
      dataFile.close();
      i++;
      }
    Serial.println("Data written to SD card.");
  } else {
    Serial.println("Error opening datalog.txt");
  }
}

void loop(void) {
// bool currentButtonState = digitalRead(BUTTON_PIN); 
  bool currentState = digitalRead(BUTTON_PIN);

  // Check for button press (transition from HIGH to LOW)
  // if (currentState == pressed) {
  // // if (lastButtonState == HIGH && currentButtonState == LOW) {
  //   recording = !recording; // Toggle recording state
  //   delay(50); // Debounce delay
  // }
  // lastButtonState = currentButtonState; // Update the last button state
  // recording = true;
  if (recording) {
    if (j == TAKE_READINGS) {
      // Save to SD card
      Serial.println("writing data block " + String(TAKE_READINGS));
      // writeToSD(TAKE_READINGS, readings);
      TAKE_READINGS = TAKE_READINGS + 10;
      if (j >= MAX_READINGS) {
        // Reached the maximum number of readings, potentially stop recording or handle as needed
        Serial.println("Maximum readings reached.");
        recording = false; // Optionally stop recording
        return;
      }
    }
    
    fsrReading = analogRead(fsrPin);
    fsrVoltage = map(fsrReading, 0, 1023, 0, 5000);
    timestamp = millis();

    if (fsrVoltage != 0) {
      fsrResistance = 5000 - fsrVoltage; // Calculate resistance
      fsrResistance *= 10000;
      fsrResistance /= fsrVoltage;

      fsrConductance = 1000000;
      fsrConductance /= fsrResistance;

      // Approximate the force
      if (fsrConductance <= 1000) {
        fsrForce = fsrConductance / 80;
      } else {
        fsrForce = fsrConductance - 1000;
        fsrForce /= 30;
      }


    }
    else {
      fsrForce = 0;
    }
          // Store the timestamp and force in the readings array
      readings[0][j] = timestamp;
      readings[1][j] = fsrForce;

      // Print current reading for debugging
      Serial.print("Timestamp: ");
      Serial.println(timestamp);
      Serial.print("FSR Force: ");
      Serial.println(fsrForce);
      j++; // Increment index for the next reading
    delay(10); // Sample delay
  }
  
}

Thanks for any help!

Welcome to the forum

You don't say which Arduino board that you are using but 2 * 10,000 long variables, each of 4 bytes requires 80,000 bytes of storage so I am not surprised that you have problems

Why is the readings array so large ?

Know that there arew some 20 different "Arduinos"! Some have a lot more memory then the basic ones. However, storing millions of mesured datasets, it migh be difficult.
Storing data, measurement by measurement, "somewhere" is the first thought. Depending on the sample rate the the best media might differ. SD card is one way that should be investigated regarding writing speed. Other memories like flash might be interesting.

I am trying to record the input force on a lever from a hand over the course of a few minutes and I need high accuracy of when, and how much force is applied to the lever.

I cannot reconcile this statement

and the size of the readings array

Hello,

I will take a look at the different Arduino's I know that mega is an option with more memory. my concern is that they will still not be enough, so I am trying to explore other options as well. The reason I am using an array is so that SD card write speeds are not a bottle neck. I don't know much about flash memory, do you think it would be a viable option for my use case?

const int MAX_READINGS = 10000; // Maximum number of readings

I would like to be able to increase the number in the code snippet above, so that I can record more readings, I used the example of a million readings in my original post. I understand that how I currently have my sketch setup this is not a possibility. If you have any suggestions on how I can record higher numbers of readings I would appreciate it!

consider moving to a Raspberry Pi 4

1 Like

The easiest is to add memory. You can add it via the SPI or I2C interface. The simplest is FRAM (Ferroelectric RAM). Both Cypress and Fujitsu offer I2C FRAM chips with densities of 4, 16, 64, 128, 256, 512, and 1024 kilobits (kb). Many of these are available on breakout boards for the Arduino and pi. They are good for trillions of cycles and are non-volatile. The 32Kx8 I use give me 8 I2C address and work at memory speed, no delay in read/write and they work on a byte by byte basis.

What is the sample rate, and bytes produced per second?

Thanks for the suggestion, I will look into that!

1 Like

Not taking into account processing time...

Sample rate: 100/second
Bytes per second (for raw data only): 800 bytes/second

That's not upsetting. Most storing technics will work.
Thinking on the system level, SD card writing gives the opportunity to read and treat data in a Pc.
Don't select a one way road with no return.

FSRs are very nonlinear and not very repeatable, so they can't be accurately calibrated.

I'm not sure if I understand what you mean?
If you say most storing technics will work do you have a suggestion for a specific technique? The system will be operating as a standalone with no connection to a PC.

Yeah, I know there are limitations to the accuracy of FSR sensors, I haven't been able to find a better alternative for my use case., unless you can any suggestions?
As I said it my original post I am recording the pressure from a hand on a lever, the lever is small only requiring one finger to pull it.

If the FSR measurements meet your requirements, fine.

For what reason do You want to store data if they will only "collect dust"?
I suppose You want to view and evaluate data later. How do You intend to do that?

Yeah, I would like to store it to analyze/graph.

Save the raw analog input reading, do the calculation later, that cuts the data down to two bytes. (even less if you want to pack four 10-bit samples into five bytes).
With a fixed sample rate, there is no need to store the time with every sample.