Read txt file into array from SD

Hi,

I am new to arduino and am a fair novice in programing.
I will try to explain what my problem is as best as possible.

I am using the arduino uno along with the arduino ethernet shield which has the micro SD slot, the ID-20 RFID reader and the I2C/TWI LCD1602 Module. I am trying to create an RFID attendance device.

The problem I am struggling with is reading a txt file from the SD card (my RFID database file) and putting each element into an array.
I have the txt file structured in the following way.

firstName lastName,IDnumber,
firstName lastName,IDnumber,

When I store the ID numbers in the program everything works fine and I can output to a txt file no problems.

What I am trying to do is read the rfid card,
load the database into memory,
check the ID against the database and store the name to a variable,
check the names of people already in attendance,
if the name is not in the attendance file add the name to the file,
output to lcd,

The code below compiles, but I do not receive the initial "ready to scan!" on the LCD and it does nothing.
If I comment out the information in the LoadRDIFdb function it will initialise with "ready to scan"

If anyone has any suggestions that would be great.
Thanks.

#include <Wire.h>
#include <LiquidCrystal_I2C.h>
#include <SdFat.h>

SdFat sd;
SdFile myFile;
using namespace std;

struct RFID
{
       char name[50];
       char ID[10];
};

const int SIZE = 25; //Lines in the txt file

RFID DB[SIZE]; //db with names and rfids
RFID CHECK[SIZE]; //class list names

// set the LCD address to 0x27 for a 16 chars and 2 line display
LiquidCrystal_I2C lcd(0x27,16,2);
// On the Ethernet Shield, CS is pin 4. SdFat handles setting SS 
const int chipSelect = 4;
//Set the RFID Reader Reset switch
int RFIDResetPin = 2;

void setup()
{
  //initialize the serial port
  Serial.begin(9600);
  
  //Enable SPI Interface
  pinMode(10, OUTPUT);
  digitalWrite(10, HIGH);
  
  //lcd initialize
  lcd.init();
  lcd.backlight();
  lcd.setCursor(1,0);
  lcd.print("Ready To Scan!"); //System Ready to scan

  pinMode(RFIDResetPin, OUTPUT);
  digitalWrite(RFIDResetPin, HIGH);
  
  // Initialize SdFat or print a detailed error message and halt
  // Use half speed like the native library.
  // change to SPI_FULL_SPEED for more performance.
  if (!sd.init(SPI_HALF_SPEED, chipSelect)) sd.initErrorHalt();
  
}

void loop()
{
  
  char tagString[13];
  int index = 0;
  boolean reading = false;
  
  //When Card presented to RFID reader
  if (Serial.available()){
  {  
    delay(100);
    // Clear Screen  
    lcd.clear();
    //Read ID
    while (Serial.available() > 0){
      //Read ID number Byte by Byte
      int readByte = Serial.read();
      
      if(readByte == 2) reading = true; //begingin of tag
      if(readByte == 3) reading = false; //end of tag
      
      if(reading && readByte !=2 && readByte != 10 && readByte != 13){
        //store the tag byte by byte
        tagString[index] = readByte;
        index++;        
        }
      }

      //Load RFIF database into memory, check tag name
      //and output data to file and LCD
      loadRFIDdb(tagString);

      }
    delay(1800);// display infor on screen for 1.8s
    clearTag(tagString); //Clear the char
    resetReader(); //Reset RFID reader
    lcd.clear(); // Clear Screen
    lcd.setCursor(1,0);
    lcd.print("Ready To Scan!"); //System Ready to scan
  }
}

void loadRFIDdb(char tag[]){
  // if there is no string exit
  if(strlen(tag) == 0) return;
  
   //read RFID DB to memory
   ifstream  sdin("RFIDDB.TXT");
   int i=0;
   do
   {
      sdin.getline(DB[i].ID,50,',');
      sdin.getline(DB[i].name,50,',');
      i++;         
   }while (i<SIZE && !(sdin.fail()));
   sdin.close();

   //check ID in DB and pull name
   boolean IdExists = false;
   char temp_name[50];
   for(int x=0; x<i; x++)
   {
          if(strcmp(DB[x].ID, tag)!=0)
          {
                strcpy(DB[x].name, temp_name);
                IdExists = true;
          }
   }
   
   //if ID was not found in the database
   if (IdExists == false){
    //Out put to LCD, Card ID Not Found
         lcd.print("ID was");
         lcd.setCursor(0,1);
         lcd.print("not found");
     return;
   }
   
    //Check Name against list allready marked in attendance
    int duplicate = readclass(temp_name);

   //if its a new student, add them to the file
   if(duplicate == 0)
   {
     lcd.print(temp_name);
     lcd.setCursor(0,1);
     lcd.print("Is Present");

     // open the file for write at end like the Native SD library
    if (!myFile.open("CLASS.TXT", O_WRITE | O_CREAT | O_APPEND)) {
      sd.errorHalt("opening test.txt for write failed");
    } 
    // if the file opened okay, write to it:
     myFile.println(temp_name);
     // close the file:
     myFile.close();
   
    return;
   }
   
   if(duplicate == 1)
   {
     //output to LCD student is already present 
     lcd.print("Student already");
     lcd.setCursor(0,1);
     lcd.print("marked present");
   return;
    }
   
}

int readclass(char Name[]){
  
   //read existing CLASS
   ifstream  sdin("CLASS.TXT");
   int i=0;
   do
   {
      sdin.getline(CHECK[i].name,50,'\n');
      i++;         
   }while (i<SIZE && !(sdin.fail()));
   sdin.close(); 

   //compare name to class list (see if its there)
   for(int x=0; x < i; x++)
   {
          if(strcmp(CHECK[x].name, Name)!=0)
          {
                return 1; //duplicate found
          } 
   }   

  return 0;

}

void clearTag(char one[]){
  //clear the array by filling with null - ASCII 0
  for(int i = 0; i < strlen(one); i++){
      one[i] = 0;
      
  }
}

void resetReader(){
  //Reset RFID reader to read again.  
  digitalWrite(RFIDResetPin, LOW);
  digitalWrite(RFIDResetPin, HIGH);
  delay(150);
}

With 60 characters per person and 25 people per array that's 1500 bytes for DB and 1500 bytes for CHECK. That is a significant portion of your memory space! I think you are running out of RAM space and crashing the Arduino.

I would suggest storing only the ID number in memory. It's not like you will have two names with the same ID. Each time you want to display a name you can look it up from the file on the SD card. You can make a function: char *name(char *ID) which loads the name into a static buffer given the ID number.

If you could store the ID number as a number (unsigned long?) rather than 10 characters you could save a significant amount of time and space.

johnwasser:
With 60 characters per person and 25 people per array that's 1500 bytes for DB and 1500 bytes for CHECK. That is a significant portion of your memory space! I think you are running out of RAM space and crashing the Arduino.

Thanks for pointing that one out for me.

Thanks for the suggestion, I'll give it a go.

Ok, so I spent about 5 hours trying to get my head around this but didn't really get too far.

I tried reading the text file line by line and doing the "if id matches" check on every line.
To achieve this I tried splitting the getline up into two arrays, one for the id and one for the name.

The problem I am getting is the following:
id - outputs the following (id,id,name)

eg. "450052FC59B2,450052FC59B2,firstname lastname ,"

it should be "450052FC59B2"

name outputs correctly
eg. "firstname lastname,"

Any ideas why the id is not being stored correctly?

ifstream sdin("RFIDDB.TXT");

char currline[40];
char id[13];

while (sdin.getline(currline, 40, '\n')){

  char name[sizeof(currline)-14];

  int i =0;
  while(i <= 13){
    id[i] = currline[i];
    i++;}
  
  i++;
  
  while(i <= 40){
    name[i-13] = currline[i];
    i++; }

  if (!myFile.open("CLASS.TXT", O_WRITE | O_CREAT | O_APPEND)) {
    sd.errorHalt();
  } 
  // if the file opened okay, write to it:
  myFile.println(name);
  myFile.println(id);
  myFile.close();

}

sdin.close();

You forgot to add a null terminator at the end of the ID. The println() function takes a character pointer and prints characters until it gets to a null character. You are also moving 14 characters (0..13) into an array sized for 13. Bad move. Change ID (and name) to contain one more character than you copy.

char currline[40];
char id[15];  // 14 characters plus  a null terminator

while (sdin.getline(currline, 40, '\n'))
    {
    char name[40-15+1];
    char *line = currline;
    int i;

   for (i=0; i < 14; i++)
      id[i] = *line++;
   id[i] = '\0';  // Make it a valid string by adding a terminator.

  line++;  // Skip the comma in the line
      
   for (int 1=0; i < sizeof name; i++)
      name[i] = *line++;
   name[i] = '\0';  // Make it a valid string by adding a terminator

  if (!myFile.open("CLASS.TXT", O_WRITE | O_CREAT | O_APPEND)) {
    sd.errorHalt();
  } 
  // if the file opened okay, write to it:
  myFile.println(name);
  myFile.println(id);
  myFile.close();

}

sdin.close();

Thanks for the quick reply. I'll give it a go when I get home.

:smiley:

I got it working thanks to your help!!
:smiley:

Only thing is it doesn't allow for a name longer than 12 characters otherwise it does something funny, but I don't have the time to troubleshoot it at the moment. I'll have to give it a break for the time being.

also I'm pretty sure you already picked up on this, but for anyone else that reads this I changed the rfiddb file to

ID,name

Thanks again John.

Only thing is it doesn't allow for a name longer than 12 characters otherwise it does something funny, but I don't have the time to troubleshoot it at the moment.

That's a restriction of the underlying fat16 format - 8 characters for the file name, a dot, and 3 characters for the extension.

I thought the code was 14 characters because that's how many you were copying. For a 12-character code:

const int IDLEN = 12;
const int NAMELEN = 27;

char currline[IDLEN + 1 + NAMELEN];  // ID, plus comma, plus Name
char id[IDLEN+1];  // ID plus a null terminator
char name[NAMELEN + 1]; // Name plus a null terminator

while (sdin.getline(currline, sizeof currline, '\n'))
    {
    char *line = currline;
    int i;

   for (i=0; i < IDLEN; i++)
      id[i] = *line++;
   id[i] = '\0';  // Make it a valid string by adding a terminator.

  line++;  // Skip the comma in the line
      
   for (int 1=0; i < NAMELEN; i++)
      name[i] = *line++;
   name[i] = '\0';  // Make it a valid string by adding a terminator

[/quote]

So I finally got some time to get back into this project and I discovered an easy solution via trial and error.

PaulS:
That's a restriction of the underlying fat16 format - 8 characters for the file name, a dot, and 3 characters for the extension.

No sorry, I meant the char name[28] array would not allow for more than 12 characters to be stored. It would cut off a name after 12.

Anyhow, I figured out an easier way to code this section.

char id [13]; //12 char plus null terminator
char name[50];

while (sdin.getline(id, 12, ',')){
    sdin.getline(name, 49, '\n');
}