How many records will there be?
You can't alter how the SD read is done. It will read 512 byte chunks from the file, or less if there is not that much data. If all the values are byte sized, you could create a binary file, with 5 bytes per record (no cr/lf needed). That would allow for 102 records before another disk read is performed.
The proper thing to do,though is perform a timing test. See if the read will be fast enough.