I'm using an ESP32, so have a bit of flash memory to work with. The data is read-only, I never need to write to it from the Arduino. I can format it however I'd like before flashing.
Two questions at the moment:
what's the best way to store that data? Doing a bit of research, I see that progmem can handle large data, and it writes to flash when the script is flashed to the ESP32. Is that the recommended way to go?
and what's the best way to look up the given day from my Arduino script? Is the best way to iterate through each line, and break when the correct date is found? Or is there some better way?
I'll be using a Real Time Clock so my Arduino will know the correct date, and the date it looks up will always be the current date.
5 years is ~1825 days so lets say 2000 days. Your example is ~20 bytes ie we have a lookuptable of "only" 40KiB. At 100Mhz+ it would be less than 1ms to find a line with your suggested simple algorithm (dont know the efficiency of clock/instruction of the ESP32).
Of course you want to store it in flash, so it stays there even with power off (ESP32 functionality depends on the board, I'm unsure if it could stay in RAM)
If you need fast access you could create an binary tree/index like structure, then you can find your date data in in microseconds.
I have no personal experience of ESP32 (yet) so the answers are not proven.
and what's the best way to look up the given day from my Arduino script? Is the best way to iterate through each line, and break when the correct date is found? Or is there some better way?
That'll do it. You could do some calendar math and figure out exactly where to find it, or some approximation that tells you where to start looking. But your way is simple and I suspect plenty fast enough, especially since you're only doing it once a day I assume.