Storing very large strings in program memory

Projects Programming
1

Hi, I want to store a very large string (167970 bytes) into the program memory (maximum of 815104 bytes). I was wondering if this is possible

I'm currently using nRF52840 that I'm programming via the arduino IDE. I'm not sure if what I'm trying to do is possible

My code is similar to this:

#include <Arduino.h>
#include <Adafruit_TinyUSB.h> // for Serial
#include <SoftwareSerial.h>

String data_storage PROGMEM = "";  // globally declared

void setup(){
   // some initial configurations here
}

void loop {
   if (Serial.available()){
      store_to_progmem();
   }
}

void store_to_progmem() {
   for (int i=0; i<50; i++) {  // each index corresponds to a 2048-byte data query
      data_storage += get_data(i); // get_data() is a function that queries a string from the web that is 2048 characters long
   }
}

String get_data() {
   String data_query;

   // this functions returns a string variable 2048 characters long 
   
   return data_query
}

for smaller String sizes, i'm still able to printout the content of the variable using this method. But for very large Strings, my code seems to timeout

2

Welcome to the forum

Can you store Strings (capital S) in PROGMEM ?

Which board are you using ?

3

Hi! I'm using an Adafruit nRF52840 Feather express. According to the PROGMEM documentation, the variables must be either globally defined, or defined with the static keyword. Any variable type is also allowed.

I'm just not sure if I'm implementing it right since my program seems to timeout. I'm assuming that its storing the data in the dynamic memory. But examples in the documentation only showed how it can be used by using constant arrays

4

Yes, because on traditional arduinos PROGMEM cant be writed by user sketch, only by bootloader or programmer.

You should read the documentation exactly for your board, is it possible to store anything in PROGMEM at the run time.

5

But a String is not a variable, it is an object created by the String library

Why not use zero terminated arrays of chars (aka C style strings - lowercase s) ?

1 Like
7

You should look up how to write to Flash not to PROGMEM

1 Like
8

that's never occurred to me before -- that an object isn't a variable

9

The nRF52840 processor doesn't have a separate address space for FLASH so the "PROGMEM" keyword does nothing. The dummy "avr/pgmspace.h" contains the lines:

#define PROGMEM
#define PGM_P const char *
#define PSTR(str) (str)

If you don't want a copy moved from FLASH into RAM, it might be sufficient to mark your string 'const'. I would use a 'const char []'.

1 Like
10

They want to store in progmem at runtime

11

Hi,

Thank you everyone for the suggestions. Based on these, I finally found a way to instead store the whole data into flash. Here's a simplified implementation:

// set buffer for receiving 2048 characters
char f_buff[2048];

// read string of length 2048
while (Serial1.available()) {
  data = Serial1.readString();
}

// convert string to character array
data.toCharArray(f_buff, 2048);       // all I needed was to convert the string into a char array so I can store to flash

// use separate function to append the 2048 data to the file
store_data_to_memory(f_buff);

// different function for storing the data into the file
void store_data_to_memory(char f_buff[2048]){
   // concatenate/store the data to file here
   // the process is also available in the nRF52840 documentation
}

I'll just repeat the process until I'm store the entire 167970 bytes into the QSPI flash as a file (limit is only 2048 per query)

12

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.