Go Down

Topic: Automate removal of SPIFFS files. Prevent SPIFFS being becoming unreadable. (Read 396 times) previous topic - next topic

Techno500

If SPIFFS becomes too full; no file is readable, would like to automate removal of first four files out of eight files.

Build list of Log filenames; then if count reaches 8 delete first four log files.

Store filenames in two dim array fileList[record][filename]

Max record will be eight.

Code: [Select]
if(record == 8)
{

for (int record = 1; record < 5; record++)
{
SPIFFS.remove("/LOG*.TXT");
}
}



if(record == 8)
{
record = 1;
}


Requesting help getting LOG filename into array; then using filename to remove file from SPIFFS

Filenames have this naming convention:  LOGxxyyzzzz.txt, xx being date, yy being month, and zzzz being year. 


William

arduino_new


UKHeliBob

Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Techno500

#3
Jul 07, 2019, 01:46 pm Last Edit: Jul 07, 2019, 04:06 pm by Techno500 Reason: added code
Quote
Requesting help getting LOG filename into array; then using filename to remove file from SPIFFS
Four LOG files will hold approximately one month of weather data.  Each Log file holds one week of data.


Apologies, for not explaining the request better.

What I have tried:

Code: [Select]
#include <FS.h>

String nameofFile[13];


int flag;
int count;

void setup()
{
 Serial.begin(115200);

  SPIFFS.begin();
 
 flag =1;
}

void loop()
{
 if(flag == 1)
 {
 listFiles();
 }
 
 if(count == 8)
 {
 deleteFiles();
 }
 
}
void deleteFiles()
{
 
 for(count = 0; count < 9; count = count +1)
 {
 Serial.println(nameofFile[count]);

 }

 Serial.println("\n\n");

 for(count =0; count < 4; count = count + 1)
 {
 SPIFFS.remove(nameofFile[count]);
 }

 Serial.println("Four files removed");
 
 listFiles();  //Confirm four files deleted

 count= 0;

 SPIFFS.end();
 
 flag = 0;

}

void listFiles()
{
 // Code to listFiles from martinayotte of the "ESP8266 Community Forum"                      
 String str;

 if (! SPIFFS.begin())
 {
 Serial.println("SPIFFS failed to mount !");
 }
 Dir dir = SPIFFS.openDir("/");
 while (dir.next())
 {
 str += "<a href=\"";
 str += dir.fileName();
 str += "\">";
 str += dir.fileName();
 str += "</a>";
 str += "    ";
 str += dir.fileSize();
 str += "<br>\r\n";
 }
 //str += "</body></html>\r\n";

 Serial.println(str);
 
}


Board is a WeMos ESP8266.


William


UKHeliBob

Presumably you want to remove the oldest file, ie the earliest one written

You have an array of filenames.  If you keep an index to the next array position to be used then you will be able to determine the index of the earliest file written so the process could be

create filename when a new one is required
close the current file
save the new filename in array at current index position
open the file for writing
increment the filename array index allowing for wrap around
SPIFFS.remove() the filename at the current array index position
repeat each time a new file is created

If you are worried about losing the value of the array index because of a reset or power outage then save that to SPIFFS too and read it, use it, increment it and write it back when required.
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Techno500

Quote
save the new filename in array at current index position
Example code?
William


UKHeliBob

Here is the basis of what I had in mind
Code: [Select]

int fileNames[] = {1, 2, 3, 4, 5, 6, 7, 8};

int fileNumber = 8;
byte arrayIndex = 7;

unsigned long currentTime;
unsigned long fileSaveTime;
const unsigned long period = 2000;  //2 second log period for illustration
char filename[12] = {"/8.LOG"};

void setup()
{
  Serial.begin(115200);
  showArray();
}

void loop()
{
  currentTime = millis();
  //write to the SPIFFS file anywhere in the main body of loop()
  //as long as you have opened the file
  if (currentTime - fileSaveTime  >= period)  //time for a new file
  {
    arrayIndex++;
    arrayIndex %= 8;
    //make full file name of file to be replaced
    sprintf(filename, "/%d.LOG", fileNames[arrayIndex]);
    Serial.print("put code here to delete ");
    Serial.print(filename);
    Serial.println(" from SPIFFS");
    //create a new filename
    fileNumber++;
    //make filename of new file to be opened and used
    sprintf(filename, "/%d.LOG", fileNumber);
    Serial.print("new filename : ");
    Serial.println(filename);
    Serial.print("adding file number ");
    Serial.print(fileNumber);
    Serial.print(" to position ");
    Serial.print(arrayIndex);
    Serial.println(" in the array");
    fileNames[arrayIndex] = fileNumber;
    fileSaveTime = currentTime;
    showArray();
  }
}
void showArray()
{
  for (int f = 0 ; f < 8; f++)
  {
    Serial.print(f);
    Serial.print("\t");
    Serial.println(fileNames[f]);
  }
  Serial.println();
}

What format will your filenames be in ?
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Techno500

I have a project that delivers two web site from one sketch:

ESP8266 Project Server


ESP8266 Domain Website

Project LOG files are stored in format "LOGxxyy.TXT"  xx being DATE and yy being the MONTH.

Project discussion and code download

Each file is stored in SPIFFS and contains Data taken at 15 minute interval for one week.

I appreciate your coding; thank you UKHeliBob!

UKHeliBob

Here is an example of storing the actual filenames as strings in an array which is closer to what yo want

Code: [Select]

char filenames[][12] =
{
  "file000.LOG",
  "file001.LOG",
  "file002.LOG",
  "file003.LOG",
  "file004.LOG",
  "file005.LOG",
  "file006.LOG",
  "file007.LOG"
};

char currentFilename[12];
unsigned long currentTime;
unsigned long fileChangedTime;
const unsigned long period = 2000;
byte arrayIndex = 7;

void setup()
{
  Serial.begin(115200);
  showFilenames();
}

void loop()
{
  currentTime = millis();
  if (currentTime - fileChangedTime >= period)
  {
    strcpy(currentFilename, filenames[arrayIndex]);
    currentFilename[6]++;
    if (currentFilename[6] > '9') //needs extending to 3 characters
    {
      currentFilename[6] = '0';
      currentFilename[5]++;
    }
    arrayIndex++;   //move to next place in the array
    arrayIndex %= 8;  //wrap round after 8 entries
    Serial.print("delete ");
    Serial.print(filenames[arrayIndex]);
    Serial.println(" at this point");
    Serial.print("new filename is ");
    Serial.println(currentFilename);
    strcpy(filenames[arrayIndex], currentFilename);
    fileChangedTime = currentTime;
    showFilenames();
  }
}

void showFilenames()
{
  for (int f = 0; f < 8; f++)
  {
    Serial.print(filenames[f]);
    if (f == arrayIndex)
    {
      Serial.println("  <<< latest entry");
    }
    else
    {
      Serial.println();
    }
  }
  Serial.println();
}
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Techno500

#9
Jul 09, 2019, 02:49 am Last Edit: Jul 09, 2019, 07:22 am by Techno500 Reason: Added code
Thank you UKHeliBob.

Thinking of using a listing of SPIFFS files to build array of "LOGxxyy.TXT" files similar to this code by techtutorialsx:

Code: [Select]
#include "SPIFFS.h"
 
void setup() {
 
  Serial.begin(115200);
 
  if (!SPIFFS.begin(true)) {
    Serial.println("An Error has occurred while mounting SPIFFS");
    return;
  }
 
  File root = SPIFFS.open("/");
 
  File file = root.openNextFile();
 
  while(file){

  if(strncmp(file.name(), "/LOG", 4) == 0)  //added to get only "LOGxxyy.TXT" files
  { 
 
        Serial.println(file.name());
 
  }

    file = root.openNextFile();
  }
}
 
void loop() {}



Could the index number be added each time the array is created; then remove first four files?

Techno500

Trying to get list of filenames and element number for filename; will remove filename based on
element number, first 4 filenames will be removed -- out of however many files are listed, with first
four characters "/LOG"

Plan to only remove files when there are eight filenames in the filelist array

Code: [Select]


#include "SPIFFS.h"

char* filelist[12][1];
char i;
byte j;
 
void setup() {
 
  Serial.begin(115200);

  if (!SPIFFS.begin(true)) { //Dev board is a ESP32
  Serial.println("An Error has occurred while mounting SPIFFS");
  return;
  }

  File root = SPIFFS.open("/");

  File file = root.openNextFile();

  while(file)
  {

   if(strncmp(file.name(), "/LOG", 4) == 0)
   {      
    
    j++;
    strcpy(filelist[i][j], file.name());  //causes a core panic
    Serial.print(filelist[i][j]);
    Serial.print("  " + j);
    }
    
    

    file = root.openNextFile();
  
  }
 
  for(j = 0;j < 4; j++)  //Delete only first four files; keep from getting too many log files.
  {
    SPIFFS.remove(filelist[i][j]);
    Serial.print("Removed:  ");
    Serial.println(filelist[i][j]);
  }
  
}
void loop() {}

  
  


Compiles; however has continuous RESET on code execution. I am less than a novice, when comes to working with arrays.

William

UKHeliBob

Code: [Select]
char* filelist[12][1];
Note that the second index level of the array allows for only 1 entry


Code: [Select]
     j++;
      strcpy(filelist[i][j], file.name());  //causes a core panic

and
Code: [Select]

  for (j = 0; j < 4; j++) //Delete only first four files; keep from getting too many log files.
  {
    SPIFFS.remove(filelist[i][j]);

What is the largest value of the second index level of the array that you are using ?

Why are you so fixated on removing 4 files at a time ?
I still think that it would be better to allow for a fixed number of files and to remove one before adding another
Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

Techno500

Four files would remove almost 30 days of data.

Plan to keep a month of data available on web server.

One file is created for each week of data.  I could count up when the files are created  and use that number to call function to listfiles to an array.  Max. files = 8.

William

Techno500

Working now...

Code: [Select]
#include "SPIFFS.h"

char* filelist[12];
int i;

 
void setup() {
 
Serial.begin(115200);

if (!SPIFFS.begin(true))
{
Serial.println("An Error has occurred while mounting SPIFFS");
return;
}

File root = SPIFFS.open("/");

File file = root.openNextFile();

while(file)
{

if(strncmp(file.name(), "/LOG", 4) == 0)
{     
   
i++;
filelist[i] = strdup(file.name());
Serial.print(filelist[i]);
Serial.print("  ");
Serial.print(i);
Serial.println("");

}

file = root.openNextFile();
}

for(i = 1;i < 5; i++)  //Delete only first four files; keep from getting too many log files.
{

SPIFFS.remove(filelist[i]);
Serial.print("Removed:  ");
Serial.print(filelist[i]);
Serial.print("  ");
Serial.print(i);
Serial.println("");

}
i = 0;
 
}
void loop() {}

 
 


William

Go Up