The problem is if you put everything into one file it will be "big". Then you will open that one file and search through it for what you want. It is this searching that will be slow.
You have to think of a clever way to do this.
For example, you could break your data down into 512 (or less) blocks - multiple files. Then use IF or SWITCH logic to figure out which file you want, and read it in quickly.
That might lead to a lot of lines of logic in your program. Maybe more memory used by that than just storing the data in memory.
How about putting each value in its own file? This would result in a lot of files, but that will not affect speed.
This would also make your program logic easier, since the value you want becomes the file's name. You can even read-in the directory of file names and do some searching through that looking for the closest match.