Ghost \n (newline)

Let’s say a .txt file that contains the following content:

A
B
C
D

and my code is as followed:

myFile = sd.open(fileName);

  while(myFile.available() > 0 && iterationPassed < iteration)
  {
    cBuffer = myFile.read();

    if(cBuffer == '\n')
    {
      if(count == 4)
      {
        Serial.print(sBuffer);
        count = 1;
        sBuffer = "";
        iterationPassed++;
      }


      else
      {
        sBuffer += "@";
        count++;
      }
    }

    else
    {
      sBuffer += cBuffer;
    }
  }
}

The string sBuffer I get is:
A
@B
@C
@D

But I want it to be:
A@B@C@D

How? Why?

ElegantBlack:
Let’s say a .txt file that contains the following content:

A
B
C
D

That is not fully describing what is really in the text file.
i.e. you are showing the letters on different lines but a text file does not have “lines” it is composed of characters.
Show the exact contents of the txt file.
i.e. perhaps something more like
AB etc…
or
AB etc…
or
AB etc…

It matters.

— bill

If the file was created on Windows the line termination is \r\n.

bperrybap:
Show the exact contents of the txt file.

That’s the content of the file. Imagine you double a file, then you see a content with four rows:
A
B
C
D

No not can be seen. (and .txt file won’t show it, right?)

For such kind of investigation the Realterm or similar terminal emulator is feasible. You can read exact bytes sent via serial line.

[quote author=Coding Badly link=msg=3580271 date=1516951172] If the file was created on Windows the line termination is \r\n.

[/quote] Not necessarily. It depends on what was used to create the file. i.e. different tools and editors can be configured to use different line endings. and a good editor will automatically recognize the line endings in the file and match it when the file is edited and re-saved.

ElegantBlack:
That’s the content of the file. Imagine you double a file, then you see a content with four rows:
A
B
C
D

No not can be seen. (and .txt file won’t show it, right?)

That is not the full contents of the file.

I guess I should have said. Show the hex value of all the bytes in the file.
i.e.
is it:
41 0D 0A 42 0D 0A 43 0D 0A 44 0D 0A
or
41 0D 42 0D 43 0D 44 0D
or
41 0A 42 0A 43 0A 44 0A

Which is it?
Or something else?

— bill

@bperrybap, you are focused on something that is trivial to make irrelevant.

@ElegantBlack, change this...

   else
   {
     sBuffer += cBuffer;
   }

...to this...

   else
   {
     if ( cBuffer != '\r' )
     {
       sBuffer += cBuffer;
     }
   }

[quote author=Coding Badly link=msg=3580294 date=1516952954] @bperrybap, you are focused on something that is trivial to make irrelevant.

[/quote]

Yea indeed, trivial..

@Coding Badly YOU ARE AWESOME :) it works. Now I understand why. Thank you

You are welcome.

[quote author=Coding Badly link=msg=3580294 date=1516952954] @bperrybap, you are focused on something that is trivial to make irrelevant. [/quote]

Based on the original post and all the subsequent posts, we simply don't know for sure where the text file comes from or the text file format. It was assumed to be in DOS format and only ever be in DOS format, but that may not necessarily be the case.

While it is possible to deal with line endings no matter what text file format was used, the OP clearly didn't understand the concept of line endings and how they can vary in "text" files. I was trying to get him to understand that line endings in a "text" file may not be a simple single character and can vary depending on how the text file was created. In DOS formatted text files, (which is most commonly used on Windows) line endings are \r\n not \n

So to fix it, the \r of DOS text file line endings had to be dealt with.

However, if the code is also needed to run on a text file that might be created using a text file format that only used carriage returns with no new lines, then the simple modification/addition done to toss carriage returns is not good enough and would fail to work as desired.

--- bill

Open file in vim.
:%!xxd

or elvis:
:display hex

file:
A
B
C
D

gives:
00000000 <41>0a 42 0a 43 0a 44 0a 0a A.B.C.D…

So here is some additional explanation about line endings that I was trying to get across, but apparently was doing a very poor job at explaining. This will explain why I was so concerned about the actual text file format.

You would think that text files are "simple" but they aren't as simple as it would appear. This is because there is more than 1 format for text files. unix created a text file format nearly 50 years ago, CP/M came along and created a slightly different one - which DOS adopted and is still used in Windows today, Apple also created their own text file format.

The difference is that each uses slightly different end of line sequences: unix uses \n DOS uses \r\n Apple uses \r

Supporting any single one is easy. Supporting unix and DOS is easy (ignore \r characters use \n as line delimiter) Supporting DOS and Apple is easy (ignore \n characters use \r as line delimiter) Supporting unix and Apple but not DOS is easy. (use either \r or \n as line delimiters) Supporting all 3 is not as easy.

So where it gets tricky is to transparently support all 3. This is because when trying to support all 3 text formats, when you encounter a \r you can't simply throw it away since you might be reading an apple text file and if so, you won't encounter a \n like in a DOS file. So when you encounter the \r you don't know if you have really reached the end of line or whether the next character will be a \n. But you can't simply accept either \r or \n as line delimiters either, since the \n after the \r causes "seeing" an empty/blank line in DOS files.

To solve this, requires a small amount of state information to ensure you don't end up processing a blank/empty line after processing a \r when you encounter a \n i.e. accept either \r or \n as line delimiters but if the previous character before a \n was a \r, ignore the \n. While not terribly difficult, it isn't as easy as just ignoring a character or accepting \r or \n as line delimiters.

So the real question for supporting text files becomes a questions of requirements. Does the s/w have to support all 3 types of text files or is unix and DOS good enough?

--- bill