Reading text from SDCard, not sure what to check for to end .read()

Hi

I am storing some default values and system state on my SDCard to read back at launch or following a power out.

The file contains a single Json string created using the ArduinoJson library.

Using the SdFat Library I am saving the file with this:

bool MyClass::saveSystemDefaults(const char * fileName, const char * data) {
 bool returnBool = true;
 if (!_file.open(fileName, O_RDWR | O_CREAT | O_AT_END)) {
    _sd.errorHalt("opening file for write failed");
    returnBool = false;
 } else {
 _file.print(data);
 }
 _file.close();
 return returnBool;
}

...and then reading the file with this:

bool MyClass::openFileAndReadData(const char* fileName) {
 bool returnBool = true;
 if (!_file.open(fileName, O_READ)) {
 _sd.errorHalt("\nERROR: Opening file for read failed");
 returnBool = false;
 }
 int data;
 unsigned int stringIndex = 0;
 char jsonData[_file.fileSize()];
 while ((data = _file.read()) >= 0) {
 Serial.write(data);
 jsonData[stringIndex] = data;
 stringIndex++;
 }
 Serial.print(F("\njsonData char*:\n"));
 Serial.println(jsonData);
 _file.close();
 return returnBool;
}

The output is almost good (the eternal optimist), but the issue is this:
The end of the output from

Serial.write(data)

is

..."relayPin":41}}}}}

But the end of the output from

Serial.println(jsonData)

is

..."relayPin":41}}}}} !jN!

What is the

!jN!

I need to take the Json string and parse it using ArduinoJson library.
I haven't implemented this yet so it may be that there is no issue but I suspect this will be a problem.

Can someone explain what these characters represent and what I might need to do to avoid them being added to my string?

I'm going to try and get a better grasp of the various functions in the SdFat Library now - but in the meantime, if there's a more appropriate / better way to do this I am all ears.

You wrote character data to the file. You are storing the characters read from the file in an int. WHY?

You wrote data from a NULL terminated array of chars to the file. When you read data from the file, you do not NULL terminated the array. Why not?

Can someone explain what these characters represent

Garbage.

and what I might need to do to avoid them being added to my string?

They are not being added to your string. They were there before you used part of the array to contain data read from the file. If you NULL terminated your array as you write data from the file into it, the print() function will know when to stop printing.

You wrote character data to the file. You are storing the characters read from the file in an int. WHY?

TBH, because I have only just tried to use the SD Card Reader and have no idea how it works. I'm simply following the example files and trying to get my head around it. It seemed to be heading in the right direction so I just kept going until I get a chance to sit down and digest the library functions.

You wrote data from a NULL terminated array of chars to the file. When you read data from the file, you do not NULL terminated the array. Why not?

It hadn't crossed my mind that this would not be done for me at the end of the file - which probably only serves to illustrate how little I understand about the language...

They are not being added to your string. They were there before you used part of the array to contain data read from the file. If you NULL terminated your array as you write data from the file into it, the print() function will know when to stop printing.

OK. Makes perfect sense to me.
So if I increase the size of jsonData

char jsonData[_file.fileSize() + 1];

and then append a zero just after the _file.read() loop

jsonData[stringIndex] = 0;

Will that do it?

It seems to make it look right at least.

I guess I need to parse it now and see what happens...

Thanks

So if I increase the size of jsonData ... and then append a zero just after the _file.read() loop ... Will that do it?

That's not exactly how I'd do it. I like to keep my NULL terminated arrays NULL terminated at all times.

char jsonData[_file.fileSize() + 1];

while ((data = _file.read()) >= 0)
{
  Serial.write(data);
  jsonData[stringIndex] = data;
  stringIndex++;
  jsonData[stringIndex] = '\0'; // Add the NULL
}

Gotcha.
Thanks again.

PaulS:
You wrote character data to the file. You are storing the characters read from the file in an int. WHY?

Because _file.read() returns an int. Otherwise it wouldn't be able to return -1 when there is no more data to read.

christop:
Because _file.read() returns an int. Otherwise it wouldn't be able to return -1 when there is no more data to read.

The only reason that that would happen would be if you failed to see that available() was non-zero before you read from the file. If you always check that available() is greater than 0, you can store the value read in a char.

PaulS:
The only reason that that would happen would be if you failed to see that available() was non-zero before you read from the file. If you always check that available() is greater than 0, you can store the value read in a char.

True, but it's not a bad way to read from a stream or file or whatever. It's exactly how it's done in (idiomatic) C and C++ with their equivalent "read" functions:

In C:

int c;
while ((c = getchar()) != EOF) {
  // do stuff with c
}

In C++:

char c;
while (std::cin >> c) {
  // do stuff with c
}

(In C++ the "std::cin >> c" part returns a reference to an object which evaluates to "false" when EOF or an error is encountered, but it's conceptually the same as a function like getchar() or Serial.read() or _file.read() which return a value that cannot occur in the input stream.)