Help looking up data in a database?

Using Arduino Project Guidance
#1

I’m trying to look up the current time in a database that’s about 1.6 megabytes large uncompressed. It has about 67k lines. The database is read only.

I’m using an Melife ESP32 board, which I gather has 4 megabytes of flash memory.

Does anyone have any tips on placing my database on that flash memory? If using Progmem, any tips on how to accomplish that?

Here’s my script:

// define our "event" struct
struct event {
  long seconds;
  char event_type; 
  float amount; 
};

// include a large db, full of events
#include "test_data.h";

// time now
long seconds_now = 1570118934;
long db_seconds;


void setup() {

  Serial.begin(115200);
  delay(10);  

}

void loop() {

  delay(1000);  

  for(int i=0; i < sizeof test_data / sizeof test_data[0]; i++) 
  {
      db_seconds = test_data[i].seconds;
      Serial.println(db_seconds);
  }  


}

And attaching test_data.h, which is the database.

test_data.h (1.6 MB)

#2

I think SPIFFS is probably the way to go on ESP32.

#3

I'm trying to look up the current time in a database ...The database is read only.

Eh?
How can you look up the current time in a read only database? The time changes all the, errr, time; a read only database never changes. It is not possible to have the current time in a read only database. You need a clock of some kind.

#4

How can you look up the current time in a read only database?

Sorry for not being clear. I'm looking up the current time in the database to see if there's a record attached to it.

#5

wrybread:
I'm looking up the current time in the database to see if there's a record attached to it.

How do you envisage relating the current time to a record in the database since, by definition, the database must have been written when what is now the current time was some distance in the future?

And if you do have a text database with 1.6MB of data wouldn't it be an awful lot easier to process it with Python on an RPi? And if the data has some structure to it you could make life easier still by using a database program like SQLite.

...R

#6

I’m looking up the current time in the database to see if there’s a record attached to it.

Do you think you could look up next week’s winning lottery numbers and perhaps send me a PM with them?

#7

wrybread:

How can you look up the current time in a read only database?

Sorry for not being clear. I'm looking up the current time in the database to see if there's a record attached to it.

Are you making an alarm clock or something triggered on time events ?

#8

[moving to a separate thread]

#9