next topic »

[on:]

Hi,

My weather station is working but I'm having issues when my internet is down. So I added an SD card to it.

I start by trying to send and if not possible I save the data to a file. When internet is back I flush SD data. It's working fine but when file reaches some size(>20 hours of 1 minute data), it last more than one minute to send one minute of data. My data is 26 bytes each minute and I'm already saving in binary instead ascii data to save space.

The "algorithm" I'm using is to send first line, erase first line(I'm opening a new file, streaming main file to temporary, remove and rename back). I don't know a better way to do this so it's a question: "Is it possible to improve this using sdfatlib beta?"

I'm trying to make additional files using some files like 0001.txt, 0002.txt with 1 hour of data each. But I'm finding a hard way dealing with ls() method. Can you post some example on dealing with this like ls, open first file, close, another file and so on?

I'm using millis() to time things. On first 5 seconds I get, try to send(timeout after 5 seconds) and if not possible, save to SD. The another 55 seconds I try to "flush" SD data. If it detects internet is down it try to send once each minute. Internet is down equals to not receive CHECKSUM from a webservice I made.

Thanks,

Wagner Sartori Junior

[Reply #1 on:]

My data is 26 bytes each minute

So, the Arduino should be able to save 10 minutes worth of data, while uploading the contents of the SD card (that was saved while the internet was not available).

Surely that's enough time to copy the whole file to the server. There would then be no reason to copy/remove/copy/rename just to remove one line from the file.

I'm trying to make additional files using some files like 0001.txt, 0002.txt with 1 hour of data each. But I'm finding a hard way dealing with ls() method. Can you post some example on dealing with this like ls, open first file, close, another file and so on?

Post a link to the library you are using. Post the code that you wrote to test the ls() method.

You could try to open hourly file. If it succeeds, that file already exists. Close it, and try the next number. When you get a failure, that file doesn't exist, so, create it and write to it.

[Reply #2 on:]

I'm trying to be safe. It's a good idea(I don't think about it before) try to send the whole file and make a simple checksum on file content to be sure the whole file is sent correctly.

I'm using http://code.google.com/p/sdfatlib/ but the beta available on http://code.google.com/p/beta-lib/downloads/list. The working code(using only one file) is available on https://bitbucket.org/trunet/weatherstation. This code is not using binary and writing everything on ASCII mode to easy debug.

The ls code is a simple root.ls(), I want to iterate on this method but I think it's not possible without hacking the library(not sure about this).

[Reply #3 on:]

The ls code is a simple root.ls(), I want to iterate on this method but I think it's not possible without hacking the library(not sure about this).

I'm not sure what you mean by "iterate on this method". The method outputs a list of files in the current directory. How can you iterate on that?

[Reply #4 on:]

I know I can't iterate. I just want a way for doing that.

Maybe getting the list and split on an array but I have no idea on how to do that.

[Reply #5 on:]

I don't think ls() is what you want. 

Maybe openNext() would be better.  It will allow you to iterate through all files in the root directory.

Here is a example from the newest version of SdFat, the 20110902 version in http://code.google.com/p/sdfatlib/.

It opens all files in the root directory and prints their names.  You could check the name and then read the file if the name matches some pattern.

Code:

/*
 * Open all files in the root dir and print their filename
 */
#include <SdFat.h>

// SD chip select pin
const uint8_t chipSelect = SS_PIN;

// file system object
SdFat sd;

SdFile file;

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  char name[13];

  Serial.begin(9600);

  // initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
  // breadboards.  use SPI_FULL_SPEED for better performance.
  if (!sd.init(SPI_HALF_SPEED, chipSelect)) sd.initErrorHalt();

  // open next file in root.  The volume working directory, vwd, is root
  while (file.openNext(sd.vwd(), O_READ)) {
    file.getFilename(name);
    cout << name << endl;
    file.close();
  }
  cout << "Done" << endl;
}
//------------------------------------------------------------------------------
void loop() {}

[Reply #6 on:]

Ow,

Thank you very much. This is what I was looking for!

Best regards,

[Reply #7 on:]

Do you know why this code is looping forever if my file only has 256K? I'm supposing that it's not detecting EOF. I'm saving files in binary mode(saving chunks of bytes) though I can't use readline().

Code:

int checksum = 0;
byte c;
while ((c = sentFile.read()) >= 0) {
  checksum ^= c;
  Serial.write(c);
}

[Reply #8 on:]

I uploaded the whole code to http://pastebin.com/ZXvZMv7t. Take a look on the sendFile() function.

I'm receiving more than 256Kb on the other side indicating the looping is happening forever.

[Reply #9 on:]

As I really don't know why the hell my while does not work(it never finds the EOF and loops forever), the code below is working flawless.

Code:

long i = 0;
byte c, checksum;
while ((c = sentFile.read()) >= 0) {
      if (i > sentFile.fileSize())
        break;
      i++;
      checksum += c;
      Serial.write(c);
} // while ((c = sentFile.read()) >= 0) {
checksum = 0xff - checksum;
Serial.write(checksum);

[Reply #10 on:]

As I really don't know why the hell my while does not work

For some reason, I read this as "my wife does not", and I was thinking you have serious communications issues. Which is funny since the whole thread is about communication failures.

[Reply #11 on:]

Could this be it?

Code:

if (i > sentFile.fileSize())

Change to:

Code:

if (i >= sentFile.fileSize())

[Reply #12 on:]

For some reason, I read this as "my wife does not", and I was thinking you have serious communications issues. Which is funny since the whole thread is about communication failures.

hahaha that was really funny!

Could this be it?

Code:

if (i > sentFile.fileSize())

Change to:

Code:

if (i >= sentFile.fileSize())

this code is working already... I want it to work without my DIY byte counter. the while loop has to work directly as the example files on library:

Code:

while ((c = sentFile.read()) >= 0) {
// do something
}