So here is some additional explanation about line endings that I was trying to get across, but apparently was doing a very poor job at explaining.
This will explain why I was so concerned about the actual text file format.
You would think that text files are "simple" but they aren't as simple as it would appear.
This is because there is more than 1 format for text files.
unix created a text file format nearly 50 years ago, CP/M came along and created a slightly different one - which DOS adopted and is still used in Windows today, Apple also created their own text file format.
The difference is that each uses slightly different end of line sequences:
unix uses \n
DOS uses \r\n
Apple uses \r
Supporting any single one is easy.
Supporting unix and DOS is easy (ignore \r characters use \n as line delimiter)
Supporting DOS and Apple is easy (ignore \n characters use \r as line delimiter)
Supporting unix and Apple but not DOS is easy. (use either \r or \n as line delimiters)
Supporting all 3 is not as easy.
So where it gets tricky is to transparently support all 3.
This is because when trying to support all 3 text formats, when you encounter a \r you can't simply throw it away since you might be reading an apple text file and if so, you won't encounter a \n like in a DOS file.
So when you encounter the \r you don't know if you have really reached the end of line or whether the next character will be a \n.
But you can't simply accept either \r or \n as line delimiters either, since the \n after the \r causes "seeing" an empty/blank line in DOS files.
To solve this, requires a small amount of state information to ensure you don't end up processing a blank/empty line after processing a \r when you encounter a \n
i.e. accept either \r or \n as line delimiters but if the previous character before a \n was a \r, ignore the \n.
While not terribly difficult, it isn't as easy as just ignoring a character or accepting \r or \n as line delimiters.
So the real question for supporting text files becomes a questions of requirements.
Does the s/w have to support all 3 types of text files or is unix and DOS good enough?