Any way to delete single lines from an SD card?

I am working on an access control keypad that stores one password per user on an SD card and have so far made it work. I am in the process of implementing wifi access to be able to download the entry logs and add users, and I believe I know how this will be done (waiting on wifi module) but the one thing I can't seem to find is a way to delete single codes from the SD card (something that would need to be done sometimes on a weekly basis). What are some ways (if it's possible at all) to go about this? I am trying to avoid having to remove the SD card if at all possible.

Copy the file omitting the lines you don't want, then rename or copy back...

Depending on how many records you need to keep and the turnover rate you could mark deleted records as such and reuse the area when a new record is added. This would require you to have fixed length records and you would need to know the maximum size of the records in order to do this.

UKHeliBob:
Depending on how many records you need to keep and the turnover rate you could mark deleted records as such and reuse the area when a new record is added. This would require you to have fixed length records and you would need to know the maximum size of the records in order to do this.

I'd say around 500 people max, with a maximum turnover of 25% yearly, all entries would have uniform size. By records do you mean the entries in a document or the documents themselves?

MarkT:
Copy the file omitting the lines you don't want, then rename or copy back...

Would you happen to know of a sample sketch that does this? I have no idea how to go about omitting the lines.

Reply #1 is the best and maybe the only way. It's also easy, but you have to do it one line at a time because there isn't room in memory for the whole file. So you have to keep the read and write file open at the same time.

Hmm... so would this be done by passing each line through a strncmp, if no match then copy it to a temp file, else skip the line and continue until the end, delete old file then copy entire temp file to a file named the same as the original, and then deleting the temp file?

By records do you mean the entries in a document or the documents themselves?

I meant the entries in a file

Each of the 500 or so people would have a record in the file, one line per person
Records available to be used for new people to be added, either never previously used or previously deleted, would have the record flagged as being available, perhaps by setting the ID, or even every data item. to a value that is impossible to enter manually.
When the need arises to add a new person you read each record in turn until an available one is found and write the new data to it, thus making it unavailable
If no record is found available to write to you add a new one to the end of the file.
To delete a record you read the file until you find it then write the dummy data items to it to make it available if/when it is needed.

ShadowfireOmega:
Hmm... so would this be done by passing each line through a strncmp, if no match then copy it to a temp file, else skip the line and continue until the end, delete old file then copy entire temp file to a file named the same as the original, and then deleting the temp file?

That is the general solution for variable length records. If the records are fixed size, you can open the file in random access mode and modify records individually.

aarg

I just realized I have no idea how to copy anything I bring up with strncmp, what's the command for that?

UKHeliBob

Your suggestion sounds like it's what I need, but I don't really know where to start. Any suggestions or examples you can throw my way?

ShadowfireOmega:
aarg

I just realized I have no idea how to copy anything I bring up with strncmp, what's the command for that?

strcpy(). All the C string functions are documented in the AVR GCC reference.

From your original post it would seem that you have much of what is needed within your reach. You can presumably write records to the SD card and read them back. Start small by writing a few test hard coded records to the SD card. 5 or 6 records will do to start with. Make one of them a record marked as deleted by storing whatever you decide to indicate that. Use a function to add each record. You can use it again later.

Now write a program to read each of them and display them on the Serial monitor. Use a function to read each record so that you can use it later. Then a program to read each record until you find a match with a hard coded record to search for. Again, wite a function to do the matching. How will you know it is the one you want ? What are you matching on ? Write your replacement "deleted" data to that record location (you already have a function to write a record) then run the program that displays all of the records again. Does everything look OK ? If not fix what is wrong before moving on.

Now to add a record. Read the file until you find a record marked as deleted and add a new one in its place. This is fundamentally the same as searching for a record as above. Write a new record in that position. Read and print the whole file again to make sure that it looks OK. Use the functions that you have already written and tested.

Up to now all of the data will have been hard coded as part of the program but now it is time to move on to getting user input as to which record should be deleted or added. Do not try this until you have got the hard coded versions working and know that they are robust. Once you have user input you can use your functions to find, add or delete a record because you know that they work.

Lets go a bit deeper into the data base you are going to create.
Are the people identified by some code, or by name only? Are the records supposed to be in some order, like ascending sequence based of a sequence number?

Some order would be nice so you would not have to search the entire data set to find one person's record.

Are you prepared to preformat entire file in the beginning, with serial number for each entry?

Have you researched the names of people that will go into the data base? Some Asian names have family name first, European have family last. Some people will have more than three names. Names may be hyphenated and quite lengthy. Some Asian family names are quite short. My Chinese daughter-in-law's father's name was "Co". Often confused with abbreviation for company!

Paul

UKHeliBob:
Use a function to read each record so that you can use it later. Then a program to read each record until you find a match with a hard coded record to search for.

This is the part I'm getting stuck on. I know how to write and read, but how do you store the records for use, and from there how do you access the stored records?

Paul_KD7HB:
Lets go a bit deeper into the data base you are going to create.
Are the people identified by some code, or by name only? Are the records supposed to be in some order, like ascending sequence based of a sequence number?

Some order would be nice so you would not have to search the entire data set to find one person's record.

Are you prepared to preformat entire file in the beginning, with serial number for each entry?

Have you researched the names of people that will go into the data base? Some Asian names have family name first, European have family last. Some people will have more than three names. Names may be hyphenated and quite lengthy. Some Asian family names are quite short. My Chinese daughter-in-law's father's name was "Co". Often confused with abbreviation for company!

Paul

People are identified by a four digit, pre-determined code.

Records are in no particular order, nor would it be feasible to put them in any order as retired codes may come back into use.

I am prepared to do what I need to to get this to work, if the end product is not convenient, it's less likely the codes will get deleted when they need to or that there will be an "accident" where all codes get deleted.

No names will go into the database.

ShadowfireOmega:
This is the part I'm getting stuck on. I know how to write and read, but how do you store the records for use, and from there how do you access the stored records?

store = write
access = read

With a 4 digit number, what is the lowest possible number and the highest possible number? The difference is the possible number of entries in your table. Format a table, one time, with all the records for all the possible numbers. How big will that table be? Mark each table entry as not used.

When you get a real entry number to look up in the table, subtract the lowest possible number from the real entry number and you will have the record number corresponding to the real entry number. Random read that record and process what ever is needed and rewrite that record.

Paul

I had to step away from this for other projects these last two weeks, coming back today I manged to get this written:

#include <SPI.h>
#include <SD.h>

File originalFile;
File newFile;
char empty = 1111;
char emptyCheck[5];
char buf[6];


void setup() {
  Serial.begin(9600);
  while (!Serial) {};
  Serial.print("Initializing SD Card...");
  if (!SD.begin(10, 11, 12, 13)) {
    Serial.println("innitialization failed.");
    return;
  }
  Serial.println("initialization complete.");
  sprintf(emptyCheck, "%04lu", empty);
  originalFile = SD.open("Here.txt", FILE_READ);
  newFile = SD.open("There.txt", FILE_WRITE);
  if (newFile && originalFile) {
    Serial.println("Here.txt and There.txt open");
    while (originalFile.available()){
      originalFile.read(buf, 6);
      if (strncmp(buf, emptyCheck, 6) == 0)
      {
        Serial.println("Found bad entry. Bad entry not copied.");
      }
      else
      {
        newFile.write(buf, 6);
        Serial.println("Found good entry. Good entry copied.");
      }
    }
  }
  newFile.close();
  originalFile.close();
}


void loop() {
  // put your main code here, to run repeatedly:

}

For some reason it is not triggering the if statement when it reads the 1111 I have in the here.txt document.

This is what the document has in it:
5684
6514
3562
1111
5985
6547
8523
1258

and after it is created, the there.txt has the same, line for line, not excluding the 1111 as I want it to. What am I doing wrong?

char empty = 1111;

A char variable is a single character. Which single character is 1111 ?

UKHeliBob:

char empty = 1111;

A char variable is a single character. Which single character is 1111 ?

It could be a single Unicode character. But that won't fit in a char.

Without looking at my ascii chart... none, as it goes to 255 >.> (if I understand your question correctly).

I have changed it to:

#include <SPI.h>
#include <SD.h>

File originalFile;
File newFile;
unsigned long empty = 1111;
char emptyCheck[5];
char buf[6];


void setup() {
  Serial.begin(9600);
  while (!Serial) {};
  Serial.print("Initializing SD Card...");
  if (!SD.begin(10, 11, 12, 13)) {
    Serial.println("innitialization failed.");
    return;
  }
  Serial.println("initialization complete.");
  sprintf(emptyCheck, "%04lu", empty);
  originalFile = SD.open("Here.txt", FILE_READ);
  newFile = SD.open("There.txt", FILE_WRITE);
  if (newFile && originalFile) {
    Serial.println("Here.txt and There.txt open");
    while (originalFile.available()){
      originalFile.read(buf, 6);
      if (strncmp(buf, emptyCheck, 6) == 0)
      {
        Serial.println("Found bad entry. Bad entry not copied.");
      }
      else
      {
        newFile.write(buf, 6);
        Serial.println("Found good entry. Good entry copied.");
      }
    }
  }
  newFile.close();
  originalFile.close();
}


void loop() {
  // put your main code here, to run repeatedly:

}

still does not work... am I doing something wrong in the if statement?

That's doing

char emptyCheck[] = "1111";

the hard way.