SD Card and parsing string problem

Dear programmer’s and hobbyist,

Today i reached a goal that ive been looking to in a long time, i did parse the string in what i wanted.
The only thing is now that if this sketch runs through the line’s that ends with \n, and it finishes the list.
It some how don’t open the file and give’s me my programmed error out of the box.
Maybe you guys have an option or solution for it?

the Code can be found below.

//--------------------------------------------------------------------------------------------------------


//--------------------------------------------------------------------------------------------------------

//DEFINES & INCLUDES
//--------------------------------------------------------------------------------------------------------
#include <SPI.h>
#include <SD.h>
#include <string.h>

#define MAX_STRING_LEN  20

char *p, *i;

File myFile;
String FNAME = "test.txt";

char line[128];
int lineCounter = 0;


int sa[6], r=0, t=0;

//SETUP
//--------------------------------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }


  Serial.print("Initializing SD card...");

  if (!SD.begin(10)) {
    Serial.println("initialization failed!");
    while (1);
  }
  Serial.println("initialization done.");
}
//--------------------------------------------------------------------------------------------------------





//LOOP
//--------------------------------------------------------------------------------------------------------
void loop() {
  // put your main code here, to run repeatedly:
  myFile = SD.open(FNAME); //Defines the File
  

  
  //Defines if opened or not.
  if (myFile)
  {
    
    while (myFile.available())
    {
      
        Serial.println("Next line");
        memset(line, '\0', sizeof(line));
        myFile.readBytesUntil('\n', line, sizeof(line) - 1);
        Serial.println(line);
         
        
        Serial.print(subStr(line, ";", 1));
        Serial.print(",");
        delay(50);
        Serial.print(subStr(line, ";", 2));
        Serial.print(",");
        delay(50);
        Serial.print(subStr(line, ";", 3));
        Serial.print(",");
        delay(50);
        Serial.print(subStr(line, ";", 4));
        Serial.print(",");
        delay(50);
        Serial.print(subStr(line, ";", 5));
        Serial.print(",");
        delay(50);
        Serial.println(subStr(line, ";", 6));
        delay(50);
        
        
      
      
        delay(1000);
    }

    myFile.close();
  }
  else
  {
    // if the file didn't open, print an error
    Serial.print("error opening ");
    Serial.println(FNAME);
  }

  delay(1000);
}
//--------------------------------------------------------------------------------------------------------





char* subStr (char* str, char *delim, int index) {
  char *act, *sub, *ptr;
  static char copy[MAX_STRING_LEN];
  int i;

  // Since strtok consumes the first arg, make a copy
  strcpy(copy, str);

  for (i = 1, act = copy; i <= index; i++, act = NULL) {
     //Serial.print(",");
     sub = strtok_r(act, delim, &ptr);
     if (sub == NULL) break;
  }
  return sub;

}

And the error message ive got is as followed.

Initializing SD card...initialization done.
Next line
123;145;164;789;154;774
123,145,164,789,154,774
Next line
143;255;214;79;4;124
143,255,214,79,4,124
Next line
153;5;44;239;24;354
153,5,44,239,24,354
Next line
176;15;765;54;445;34
176,15,765,54,445,34
error opening test.txt
error opening test.txt
error opening test.txt
error opening test.txt

Really hope you guys might have an solution for it! cheers.

anyone?

It some how don't open the file and give's me my programmed error out of the box.

What do you want the program to do if it can't open the file ?

This might not have anything to do with your problem, but the original SD library example sketch on which your sketch seems to be based, has everything in setup(). (There's no need to use a while() in loop(); the example code just has a while() in setup() and an empty loop().)

So in the library example, the file is only opened once, in setup(). You have the open in loop(), so when the while has finished doing its thing after the first open, it goes back to the top of loop() and opens (or evidently tries to, but fails to) the file again. (I have no idea why that might not be allowed, but there may be some nuance there....)

So just stick everything in setup(), or at least put the open there and see if that helps?

UKHeliBob:
What do you want the program to do if it can't open the file ?

I want it to try and open the file again. At least start at top of the list again.

One reason for failure to open a file is lack of memory. Open allocates a 512 byte buffer using malloc. Grab a copy of the freemem function (Adafruit has one) and see how much space you have before the open.

IIRC, strtok uses malloc too, so you may be seeing some fragmentation.

Where my theory falls apart is that since you close the file, the buffer should be available again for the second (failing) attempt. Still worth checking though.

If that's not it, I'd take a copy of the library and put some prints in it to find out which reason it had to say that it couldn't open the file.

Twinkleyan, I wanted to react on your post as well with the following post. But I've got an 5 min pause to it somehow haha.

I'm going to try that one! But the thing is that eventually I have to save inputs from the Txd and rxd pins of the arduino. That's why I need the sd.close() and the sd.open() functions. Because when you call sd.close() it saves the file and closes the connection.
If they had a function just like save() that would solve this problem. Haha unfortunatly there isn't one yet.

Well thanks for your reply!

wildbill:
One reason for failure to open a file is lack of memory. Open allocates a 512 byte buffer using malloc. Grab a copy of the freemem function (Adafruit has one) and see how much space you have before the open.

IIRC, strtok uses malloc too, so you may be seeing some fragmentation.

Where my theory falls apart is that since you close the file, the buffer should be available again for the second (failing) attempt. Still worth checking though.

If that's not it, I'd take a copy of the library and put some prints in it to find out which reason it had to say that it couldn't open the file.

Hmm good point! Going to dive in to that this weekend! Going to let you know asap!

I want it to try and open the file again.

Put the attempt to open the file in a while loop with a condition that only exits if the file is successfully opened and/or a count of the number of attempts reaches some limit that you decide. Increment the count and print a progress indicator (maybe a full stop) each time an attempt is made. Use a short delay() between each attempt or if the program is required to be doing something else whilst attempting to open the file then use millis() for non blocking timing. What should happen if the program fails to open the file after say 100 attempts ?

I'm going to give that a shot as well, and if it's failing to load. It needs to reboot the arduino, but I think that is not possible from the arduino itself. Only external I guess. Well in going to dive in to it a bit deeper this weekend thanks for your reply!!

It needs to reboot the arduino

Why ?

I found out that it didnt clear the memory of the Nano, How is it possible to clear the memory? because it try's to load into the file, while it doesnt have any line's to read after four lines. this means when the arduino has read all the line's of the .txt file it will stop and won't open the file anymore.

And the reason why i need to reboot the arduino is when it fails it has to clear all his memory and data and trying to catch up again.

It looks like your combination of myFile.available() and readBytesUntil() may be screwing things up. Once there are no more bytes available the program should close the file and release the memory used.

Does the program ever close the file ? Put a print statement after the close() so that you know that it happens

Just tried what you mentioned. and it print's out that it closes the file! so that part works

I doubt if you printing out "yay the file closed" or similar as the line after the myFile.close(); is any proof that the file actually closed.

The only way to know would be to test myFile.close()'s return value, and since there isn't one, you can't.

From the reference page:

close()

Close the file, and ensure that any data written to it is physically saved to the SD card.

Syntax
file.close()

Parameters
file: an instance of the File class (returned by SD.open())

Returns
none

Somehow i found a thing that actually resets the Nano, i need some reviews on it so i know if it's going to be stable enough.

void loop(){
  software_Reset();
}


void software_Reset() // Restarts program from beginning but does not reset the peripherals and registers
{
  asm volatile ("  jmp 0");  
}

And i know it would't return anything, that's also the reason i can't know that it's closing the .txt file.
But what i know is that it reached that point so far.

j_molenaar:
But what i know is that it reached that point so far.

You knew that all along. The only way it could have given you this spew:

error opening test.txt
error opening test.txt
error opening test.txt
error opening test.txt

... was for your while() to have finished (and the close is the very next line) and for the if() to have failed next time round. It can't possibly have got there without having tried to close the file.

That is either good news or bad news. I don't know which, but I can't see the problem.

Your program uses a curious mixture of String (uppercase S) and string (lowercase s) functions. Why not use one or the other ?

i know, but how could i found out that it closes it.
And well that might be bad news for me haha, can't get it to work right.
And how you mean? i only see upper case String cases. haha

Ok here’s some testing I did, to see if in principle a file can be opened, used, closed and then opened, used, closed a few times. And yes it can.

This is the content of my file after creating and re-opening in Windows text editor:

line1 of 2
line2 of 2

The code below is basically DumpFile from the SD card library. It has a few serial prints to show what’s happening, but the biggest change is that the file is dumped more than once due to a for() loop of the original code in setup().

/*
  BASED ON SD card file dump

  This example shows how to read a file from the SD card using the
  SD library and send it over the serial port.
           **** this version dumps the file a few times.... ****

  The circuit:
   SD card attached to SPI bus as follows:
 ** MOSI - pin 11
 ** MISO - pin 12
 ** CLK - pin 13
 ** CS - pin 4 (for MKRZero SD: SDCARD_SS_PIN)

  created  22 December 2010
  by Limor Fried
  modified 9 Apr 2012
  by Tom Igoe

  This example code is in the public domain.

*/

#include <SPI.h>
#include <SD.h>

const int chipSelect = 4;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  Serial.print("Initializing SD card...");

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    while (1);
  }
  Serial.println("card initialized.");

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  for (byte i = 1; i < 4; i++)
  {
    Serial.print("Opening the file, count is ");
    Serial.print(i);
    File dataFile = SD.open("text.txt");
    Serial.println("... open (as far as I know)");
    // if the file is available, read from it:
    if (dataFile)
    {
      while (dataFile.available()) {
        Serial.write(dataFile.read());
      }
      Serial.println(" ");
      Serial.print("     Closing the file");
      dataFile.close();
      Serial.println("... closed (as far as I know)");
    }
    // if the file isn't open, pop up an error:
    else
    {
      Serial.println("error opening text.txt");
    }
  }//for
}

void loop() {
}

Here’s the output:

Initializing SD card...card initialized.
Opening the file, count is 1... open (as far as I know)
line1 of 2
line2 of 2 
     Closing the file... closed (as far as I know)
Opening the file, count is 2... open (as far as I know)
line1 of 2
line2 of 2 
     Closing the file... closed (as far as I know)
Opening the file, count is 3... open (as far as I know)
line1 of 2
line2 of 2 
     Closing the file... closed (as far as I know)

What does this prove? It proves there is no inherent problem or limitiation or whatever that prevents a file being opened, used, closed a few times.

Suggestion to OP: run that code with your file and see what happens.

If it doesn’t work, report back.
If it works, report back.