Where & How to store a lookup table

So... I want to use a 256 byte lookup table in a project. I have a number of options on how and where to store it and I'm trying to figure which is the best approach.

The lookup table will be read-only while the controller is using it. However it will need to be occasionally updated. My project includes an SD card for logging, so I can put a file on there with the data, and update the file on the card when necessary.

I considered doing lookups directly against the file, but I don't see any way with the SD library to do random-access byte-by-byte reads of a file. If I wanted, say, byte #203, is there a way to quickly get just that byte without reading thru the whole file up to that point? There will be lots of lookups and speed is an issue.

I considered reading it from that file and putting it in an array in regular ram. This is the simplest approach (KISS). I'm doing this project on a Mega and although I'm using a good bit of the ram already, I still have plenty left. It's simple enough, and would certainly be fast. Just seems to me like poor programming. It goes against many of the rules I've been taught about efficient use of resources on microcontrollers. It's conceivable that I may need that ram for other things later.

Likewise, I considered reading it from the file and putting it into EEPROM. I can read it from EEPROM with a base address + an offset. To reduce boot time and wear on the EEPROM, I would read the file from the card only if it exists and put it into EEPROM then delete or rename the file. I know EEPROM is slow to write, but I think it reads pretty quickly doesn't it?

Then finally, I've thought about how I might put it into flash. My research seems to indicate that writing to flash at runtime is complex, and requires functions in the bootloader. So reading a new file from the SD card and having the controller place it into flash does not seem practical???

Next I thought about how I might be able to have that data in a separate file on my computer, and somehow include it when I compile. The included file would have to be in a format easily created by Excel which is where the master copy of the data is stored and updated. I can create a csv, or even raw binary, but having Excel create a c++ file would involve some sophisticated macros.

I would love to hear how you all think I should approach this. Can someone give me an overview of how I might be able to do quick random access on the file on the SD card, or maybe how I could easily include this data with my compile? Or other techniques? I always love learning a new trick!

(deleted)

You can SEEK to the data you want in an SD.

DrWizard:
how I might be able to do quick random access on the file on the SD card,

Having read all the possible avenues in your post I finally came to the conclusion: why bother?

256 bytes is small(ish) so you might as well just read it in from SD card and then access it from within the Arduino at run-time. Let's face it, you could read that amount of data, in memory, serially in a few milliseconds.

It's almost getting to the point of over-engineering something do put 256 bytes into EEPROM or keeping it up-to-date from SD card with all the sync issue to worry over.

Now, if you were talking about 256Kb that would be a different matter!

Edit: just read the bit where you had considered this on a MEGA of all things! Yes, KISS. Always! Better to surrender 256 bytes of memory than 50 hours of programming effort on any of the other options.

I think a global 2D array is the way forward?

Also look up what "binary searching" is. Itll be quicker than reading each successive array entry ill you get the one you want.

If memory usage becomes a problem, use a formula.
See LINEST, in EXCEL.

.

Johnny010:
I think a global 2D array is the way forward?

Why 2D?

Also look up what "binary searching" is. Itll be quicker than reading each successive array entry ill you get the one you want.

Only of the data can be ordered.

I use EEPROM to store values that may need "adjustment" at a later date. Simple interface and easy to use, minus the miles of code some suggest.

Bill

AWOL:
Why 2D?
Only of the data can be ordered.

If the x values are the original and y are the values that will replace x, then surely a 2D array is needed?

Eg. x -> y 126216 -> 929129

You are assuming the position in the array is equal to x?

And secondly, seeming as all data is binary and can be represented numerically...how can it NOT be ordered?

He says it is "Read Only" with occasional edits. So I think it is safe to assume that ordering the data would be useful....

*PS Edit - Ah I see. If it is 256 bytes of array, then seeming as the array is holding only bytes, then if there is no repeat of x, then each array position = x-1.