Pages: 1 [2] 3 4   Go Down
Author Topic: List only *.csv files  (Read 3022 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

okay I think I have figured out something but not too sure as I am running a w5100 card with the SD I have disabled the network part on start up

well I know now it is only an issue with creating it stop at

error: Write failed

should I change the line to something else?

Code:
#include <SdFat.h>
#include <SdFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
SdFile plFile; // Playlist file
SdFile lFile; //list files
      uint16_t count; // count of files
        int  FNAME_SIZE;
        char  fname[12];
        char  file_list[] = "FILELIST.TXT";
        char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
      pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();
Serial.println("program started");
if (plFile.open(file_list, O_CREAT|O_WRITE)){
    sd.errorHalt("Write failed");
  }
 
 count = 0;
 while (file.openNext(sd.vwd(), O_READ))
{
  Serial.println("Started");
    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.print(fname);
         plFile.write(fname, 12);
count++;
}
}
file.close();

}
plFile.close();
Serial.println("count");
Serial.print(count);
}

void loop() {}
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 653
Posts: 50883
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
SdFat sd;
SdFile file;
SdFile plFile; // Playlist file
SdFile lFile; //list files
      uint16_t count; // count of files
        int  FNAME_SIZE;
        char  fname[12];
        char  file_list[] = "FILELIST.TXT";
        char *CSV_EXT = "CSV";

What's up
                   with the bizarre
                   indenting? Haven't
                   you been introduced to
                   Tools + Auto Format?

If not, you should learn how to use it.
Logged

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
with the bizarre indenting

it was from the original code from marco that I have changed to get working with what I need, I have cleaned up the code a bit more.

and I have tried to change the way it logs to a file separately they both work but together they don't instead of writing the list of files to a file can I write it to a buffer?

Code:
#include <SdFat.h>
#include <SdFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
//SdFile lFile; //list files
uint16_t count; // count of files
int FNAME_SIZE;
char fname[12];
char file_list[] = "FILELIST.TXT";
char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
pinMode(10,OUTPUT);
digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

Serial.println("program started");

ofstream sdlog(file_list ,  ios::out | ios::app);

 

 count = 0;
 while (file.openNext(sd.vwd(), O_READ))
{
  Serial.println("Started");
    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.print(fname);
           sdlog << (fname,12) << endl;
count++;
}
}
file.close();

}
Serial.println("count");
Serial.print(count);







 if (!sdlog) sd.errorHalt("append failed");


sdlog.close();
}
 


void loop() {}
Logged

Sydney, Australia
Offline Offline
Edison Member
*
Karma: 33
Posts: 1287
Big things come in large packages
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

A few pretty simple questions:
1. Do you actually have any csv files on the SD card?
2. Can you read the SD card on a computer? Is it formatted? Is the write protection off?
3. Have you tried any of the example programs that come with the libraries. There is usually at least one that displays the files on the SD card? If you run those, do they display files?
4. Are you sure that there are no hardware conflicts (eg, Arduino pins for chip select)?

The odd formatting from my code seen when using the IDE comes from the fact I don't use the Arduino IDE and the editor uses real tabs. Just reformat as required.

Quote
and I have tried to change the way it logs to a file separately they both work but together they don't instead of writing the list of files to a file can I write it to a buffer?

I would suggest that you start by writing the file names to the serial console. Once you have mastered that you can write them anywhere.
« Last Edit: February 18, 2013, 08:40:07 pm by marco_c » Logged

Arduino libraries http://arduinocode.codeplex.com
Parola hardware & library http://parola.codeplex.com

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

if I run this code this is what I get with out creating the file so it is doing what it is meant to but just not writing it to a file.

program started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
3V_FILE.CSVStarted
Started
Started
TEST.CSVStarted
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
Started
10000101.CSVStarted
10000219.CSVStarted
10000308.CSVcount
Code:
#include <SdFat.h>
#include <SdFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
//SdFile lFile; //list files
uint16_t count; // count of files
char fname[12];
char file_list[] = "FILELIST.TXT";
char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
pinMode(10,OUTPUT);
digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

Serial.println("program started");

//ofstream sdlog(file_list ,  ios::out | ios::app);

 

 count = 0;
 while (file.openNext(sd.vwd(), O_READ))
{
  Serial.println("Started");
    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.print(fname);
               
           //sdlog << (fname,12) << endl;
count++;
}
}
file.close();

}
Serial.println("count");
Serial.print(count);







 //if (!sdlog) sd.errorHalt("append failed");


//sdlog.close();
}
 

Logged

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

okay I worked it out it was the sdfile reference it was meant to be SdFileBase all working and writing to the log file the last issues is it only seems to be logging the first name and a bit

Code:
#include <SdFat.h>
#include <SdFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
SdBaseFile lFile;
uint16_t count; // count of files
char fname[12];
char file_list[] = "FILELIST.TXT";
char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
pinMode(10,OUTPUT);
digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

Serial.println("Program started");

if (lFile.open(file_list, O_CREAT|O_WRITE));

 count = 0;
 while (file.openNext(sd.vwd(), O_READ))


    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
   Serial.print("File added ");             
  Serial.println(fname);
                lFile.write(fname, 12);
           //sdlog << (fname,12) << endl;
count++;
}
}
file.close();

}
  Serial.println("");
Serial.print("File Count ");
Serial.print(count);
}
 


void loop() {}
Logged

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

no idea what I have done but this is what is getting logged now

嘳䙟䱉⹅千V䕔呓䌮噓堀T〱〰㄰㄰䌮噓〱〰㈰㤱䌮噓〱〰㌰㠰䌮噓

Serial print out

program started
3V_FILE.CSV
TEST.CSV
10000101.CSV
10000219.CSV
10000308.CSV
Count 5

current code
Code:
#include <SdFat.h>
#include <SdFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
SdBaseFile lFile;
uint16_t count; // count of files
int const file_size =12;
char fname[file_size];
char file_list[] = "FILELIST.TXT";
char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
pinMode(10,OUTPUT);
digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

Serial.println("program started");

if (lFile.open(file_list, O_CREAT|O_WRITE));

 count = 0;
 while (file.openNext(sd.vwd(), O_READ))

    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.println(fname);
                lFile.write(fname, file_size);
          count++;
}
}
file.close();

}
lFile.close();

Serial.print("Count ");
Serial.print(count);

}
 


void loop() {}
Logged

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

okay I have got a bit further now it is logging all the filename
it ended up to do with
Code:
lFile.write(fname, 12);

if the file is not 12 characters it will stuff it up so I got rid of the shorter name files now am a stuck to how to get it to write a new line instead of putting them all in a line any ideas?

10000101.CSV10000219.CSV10000308.CSV

Code:

#include <SdFat.h>
#include <SdFile.h>
#include <SdBaseFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
SdBaseFile lFile;
uint16_t count; // count of files
int const file_size =12;
char fname[file_size];
char file_list[] = "FILELIST.TXT";
char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
pinMode(10,OUTPUT);
digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

Serial.println("program started");

if (lFile.open(file_list, O_CREAT|O_WRITE));

 count = 0;
 while (file.openNext(sd.vwd(), O_READ))

    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.println(fname);
                lFile.write(fname, 12);
          count++;
}
}
file.close();

}
lFile.close();

Serial.print("Count ");
Serial.print(count);

}
 


void loop() {}
Logged

Sydney, Australia
Offline Offline
Edison Member
*
Karma: 33
Posts: 1287
Big things come in large packages
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

'\n' is the newline character.

Serial.print("Hello\n");

is the same as

Serial.println("Hello");
Logged

Arduino libraries http://arduinocode.codeplex.com
Parola hardware & library http://parola.codeplex.com

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

okay I have almost got everything I need working, I have removed the need to log the information to file and got it to processes straight away to an ftp now I just need to check if the file being processed matches a set file name and skip it or if not delete it

 this is the bit of code I am have issues with so what I what I am trying to resolve is if the fname = "10000219.CSV" skip else delete the file
Code:
int const file_size =13;
char fname[file_size];
char dname[file_size];
char file_list[13] = "10000219.CSV";



 if (fname == file_list)
               {
                Serial.print ("Todays File");
               }
               else
               {
               dfile.open(fname);
               dfile.remove();
                Serial.print("File Removed");
          }


this code works a treat it uploads each file one at a time.



Code:
/*
   FTP passive client for IDE v1.0.3 and w5100/w5200
   Posted orginally posted by SurferTim re Edited by Onenate 18/02/2013 works with SDFat libary
   orginal code only works with sd.h libary remove download option only upload to server
   http://playground.arduino.cc//Code/FTP
*/

#include <SdFat.h>
#include <SdFatUtil.h>
#include <SdFile.h>
#include <SdBaseFile.h>
#include <SPI.h>
#include <Ethernet.h>

// comment out next line to write to SD from FTP server
#define FTPWRITE
#define error(s) error_P(PSTR(s))

// this must be unique
byte mac[] = { 0x90, 0xA2, 0xDA, 0x00, 0x59, 0x67 }; 

// change to your network settings
IPAddress ip( 192, 168, 3, 177 );   
IPAddress gateway( 192, 168, 3, 254 );
IPAddress subnet( 255, 255, 255, 0 );


// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
SdBaseFile lFile;
uint16_t count; // count of files
int const file_size =13;
char fname[file_size];
char dname[file_size];
char file_list[13] = "10000219.CSV";
char *CSV_EXT = "CSV";
#define SD_SELECT 4


/************ SDCARD STUFF ************/
Sd2Card card;
SdVolume volume;
SdFile root;
SdFile dfile;
SdBaseFile ufile;
SdFat remove;

// change to your server
//IPAddress server(serverName);
char serverName[] = "update.ftp.com.au";

EthernetClient client;
EthernetClient dclient;

char outBuf[128];
char outCount;

// change fileName to your file (8.3 format!)
//char fileName[13] = "10000219.CSV";


ArduinoOutStream cout(Serial);
//...............errors..................
void error_P(const char* str) {
  PgmPrint("error: ");
  SerialPrintln_P(str);
  if (card.errorCode()) {
    PgmPrint("SD error: ");
    Serial.print(card.errorCode(), HEX);
    Serial.print(',');
    Serial.println(card.errorData(), HEX);
  }
  while(1);
}

void setup()
{
  Serial.begin(9600);

  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  if(sd.begin(4) == 0)
  {
    Serial.println("SD init fail");         
  }

Serial.println("\nFiles found on the card (name, date and size in bytes): ");
  //root.openRoot(volume);
 
  // list all files in the card with date and size
  root.ls(LS_R | LS_DATE | LS_SIZE);


  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  digitalWrite(10,HIGH);
  delay(2000);
  Serial.println("Ready. Press f ");
}

void loop()
{
  byte inChar;

  inChar = Serial.read();

  if(inChar == 'f')
  { callfile();
    Serial.println("FTP OK");
     }

  }
 
 
 
 
void callfile()
{
 count = 0;
 while (file.openNext(sd.vwd(), O_READ))

    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.println(fname);
                doFTP();
               if (fname == file_list)
               {
                Serial.print ("Todays File");
               }
               else
               {
               dfile.open(fname);
               dfile.remove();
                Serial.print("File Removed");
          }

                count++;
}
}
file.close();

}
}

byte doFTP()
{


  Serial.println("SD opened");

  if (client.connect(serverName,21)) {
    Serial.println("Command connected");
  }
  else {
    ufile.close();
    Serial.println("Command connection failed");
    return 0;
  }

  if(!eRcv()) return 0;

  client.write("USER username\r\n");  // change your username here

  if(!eRcv()) return 0;

  client.write("PASS password\r\n"); // change your password here

  if(!eRcv()) return 0;

  client.write("SYST\r\n");

  if(!eRcv()) return 0;

  client.write("PASV\r\n");

  if(!eRcv()) return 0;

  char *tStr = strtok(outBuf,"(,");
  int array_pasv[6];
  for ( int i = 0; i < 6; i++) {
    tStr = strtok(NULL,"(,");
    array_pasv[i] = atoi(tStr);
    if(tStr == NULL)
    {
      Serial.println("Bad PASV Answer");   

    }
  }

  unsigned int hiPort,loPort;

  hiPort = array_pasv[4] << 8;
  loPort = array_pasv[5] & 255;

  Serial.print("Data port: ");
  hiPort = hiPort | loPort;
  Serial.println(hiPort);

  if (dclient.connect(serverName,hiPort)) {
    Serial.println("Data connected");
  }
  else {
    Serial.println("Data connection failed");
    client.stop();
    ufile.close();
    return 0;
  }

  Serial.println("open file");
 ufile.open(fname,O_READ);

  client.write("STOR ");
  client.println(fname);

  if(!eRcv())
  {
    dclient.stop();
    return 0;
  }

  Serial.println("Writing");
  byte clientBuf[64];
  int clientCount = 0;

  while(ufile.available())
  {
    clientBuf[clientCount] = ufile.read();
    clientCount++;

    if(clientCount > 63)
    {
      Serial.println("Packet");
      dclient.write(clientBuf,64);
      clientCount = 0;
    }
  }

  if(clientCount > 0) dclient.write(clientBuf,clientCount);


  dclient.stop();
  Serial.println("Data disconnected");

  if(!eRcv()) return 0;

  client.write("QUIT\r\n");

  if(!eRcv()) return 0;

  client.stop();
  Serial.println("Command disconnected");
 
  ufile.close();
  Serial.println("SD closed");
  return 1;
}




byte eRcv()
{
  byte respCode;
  byte thisByte;

  while(!client.available()) delay(1);

  respCode = client.peek();

  outCount = 0;

  while(client.available())
  { 
    thisByte = client.read();   
    Serial.write(thisByte);

    if(outCount < 127)
    {
      outBuf[outCount] = thisByte;
      outCount++;     
      outBuf[outCount] = 0;
    }
  }

  if(respCode >= '4')
  {
    efail();
    return 0; 
  }

  return 1;
}


void efail()
{
  byte thisByte = 0;

  client.write("QUIT\r\n");

  while(!client.available()) delay(1);

  while(client.available())
  { 
    thisByte = client.read();   
    Serial.write(thisByte);
  }

  client.stop();
  Serial.println("Command disconnected");
  ufile.close();
  Serial.println("SD closed");
}

Logged

Sydney, Australia
Offline Offline
Edison Member
*
Karma: 33
Posts: 1287
Big things come in large packages
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

This
Code:
(fname == file_list)

will never do what you expect it to do. You are comparing if the addresses (pointers) of the two strings are the same.

If you want to compare whether the strings are the same you need to use the strcmp() function, like you were in previous code to compare the file extension.
Logged

Arduino libraries http://arduinocode.codeplex.com
Parola hardware & library http://parola.codeplex.com

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

thanks for that works a treat

Code:
    if (strcmp(fname,file_list) == 0)

now I just need to get it to delete the file it just does not seem to work with anything I try if, I open the file then delete it, or just try and delete it still no luck. 


Code:

dfile.open(fname);
               dfile.remove();
                Serial.print("File Removed");
Logged

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

okay if I add the remove file command it does not complete the files are still their, it also stuff up the uploading of the file

but I need to remove these files some how.

any ideas

Logged

Sydney, Australia
Offline Offline
Edison Member
*
Karma: 33
Posts: 1287
Big things come in large packages
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Have you tried any of the examples for the libraries to see how you can use remove()
Logged

Arduino libraries http://arduinocode.codeplex.com
Parola hardware & library http://parola.codeplex.com

Offline Offline
Full Member
***
Karma: 0
Posts: 104
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

yer I have checked out the remove commands and everything I have tried nothing seems to deleted it ,

Another strange issue it does not seem to loop it only list the files ones then that it any ideas why or what I should change?

Code:
#include <SdFat.h>
#include <SdFile.h>
#include <SdBaseFile.h>

// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
SdFile file;
SdBaseFile lFile;
uint16_t count; // count of files
int const file_size =12;
char fname[file_size];
char file_list[] = "FILELIST.TXT";
char *CSV_EXT = "CSV";
#define SD_SELECT 4

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
           
pinMode(10,OUTPUT);
digitalWrite(10,HIGH);
 
 
if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

Serial.println("program started");


}
 


void loop() {
delay(500);
count = 0;
 while (file.openNext(sd.vwd(), O_READ));

    if (file.isFile())
{ file.getFilename(fname);
if (strcmp(CSV_EXT, &fname[strlen(fname)-strlen(CSV_EXT)]) == 0){
                Serial.println(fname);
                count++;
}
}
file.close();

}
count = count = 0;
}
Logged

Pages: 1 [2] 3 4   Go Up
Jump to: