Show Posts
Pages: 1 2 [3]
31  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: February 03, 2009, 04:58:29 pm
You guys are AMAZING!!  smiley-grin

I've only had time this morning to quickly compile & try bobemoe's Print_Data example, but I can see that it's EXACTLY what I was after. I should be able to incorporate this code to 'format' the data stream in the most user-friendly way possible, which will make life so much easier for my workmates (and me!).

I'll be away from my computer today so I won't be putting much time into this, but in the next couple of days I'll let you know exactly how things have gone.

Charlie & bobemoe - you're work here will be extremely valuable to many others like myself I'm sure; we can't thank you guys enough.

Oh, one last thing - Charlie, did you have any thoughts on why my RTC data is being 'corrupted'?

Thanks again everyone!!

JB
32  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: February 02, 2009, 11:32:14 pm
Thanks for explaining the weird characters, your post is very informative!!

Unfortunately (for me) I'm using Windows... don't get much choice with the Government!!  ;D

I guess that's another option to look at; does anyone know of an existing Windows program which could convert the raw data to CSV or just plain text? I had hoped to have been able to just pull the SD card out, plug it into the laptop & view the output in 'real' numbers, but I now understand that I can't do that currently. I can't get over how generous the Arduino community is in terms of writing & sharing code with people like me, and I would be deeply grateful to have a look at a writeString() function if you were to play around with one!

Since my data is only taken once a minute, and the capacity of SD cards provides essentially unlimited storage in my case, speed and file size are not really an issue. Your comment:
Quote
In your case trying to figure out this string thing (its actually an array of chars) might be an idea.
...has got me thinking though, so I'll start playing around with that & see where it takes me...

Thanks again!
33  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: February 02, 2009, 08:35:18 pm
Downloaded & tried, thanks bobemoe!

Here is what is in my data.bin file after running the sketch:


-< edit >- bugger - how do I put a photo in here?!? There should be a bmp of strange shapes & symbols instead of numbers here...

Do you know why I'm getting this wierd stuff instead of numbers?

Thanks for your help anyway!!

JB
34  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: February 02, 2009, 06:23:18 pm
Hi Charlie,

I'ts probably more accurate to say that 'my' code is exposing about half a dozen forum poster's souls - there's not too much in there that I've written myself but rather cobbled together from this wonderfully supportive community!  smiley

The original rundown of my project was as follows:
  • A remote, solar-powered, automated camera system to take photos of key wetlands at specified times each day
  • Using a Canon Powershot A720 consumer camera, with modified firmware (see http://chdk.wikia.com/wiki/CHDK for more info - I'm sure there are people here who would be interested in what the CHDK community has achieved)
  • Thermistor to monitor enclosure temperature, and operate fans when required
  • DS1307 mini-board (http://www.futurlec.com/Mini_DS1307.shtml) for RTC
  • Arduino to control fans, camera power & solenoid to turn camera on at specific times
  • Camera should happily run unattended for around 3 months
So, basically the sketch as originally conceived was designed to:
  • monitor the time from the RTC, and the temperature from the thermistor.
  • check the temperature against the previous second's readings; if it is higher than the MaxTemp, make this reading the new MaxTemp; if it is lower than the MinTemp, make it the new MinTemp.
  • at midnight, record the date & daily MaxTemp to EEPROM & reset MaxTemp to 0; around midnight, record the daily MinTemp & reset to 40.
  • if the temperature is over 35 degrees C, start the fans; if the temp has dropped below 33 degrees 1 minute later, switch the fans off, otherwise leave them running & keep checking each minute.
  • check to see if the time matches any of the pre-determined times to take a photo.
  • If it's time for a photo: (1) record whether the fans are on or off, and switch them off if they are on; (2) send power to the camera; (3); trigger the solenoid for 100mS, which pushes the camera's power button; (4) leave power on to the camera for 15 seconds so the camera can take a photo then power itself off; (5) return fans to their previous status.
However, on implementing a prototype system, we have found that the enclosure & heat-shield we've employed makes the temperature control functionality redundant - the interior temperature is the same as the external temp, so the fans don't actually provide any cooling effect. I'll now use them to just blow air over the camera aperature a few times a day, in order to remove dust and/or insects.

We thought that it would be helpful to record daily maximum & minimum temperatures, so I set up a sketch which would record the date (dd/mm/yy) & max & min temps in 5-byte chunks once each day in EEPROM. The data dump procedure was initially set up to read the EEPROM & display it in a table-like output like this:
         Data dump initiated:

   Date      MaxTemp      MinTemp
1/1/2009         20         12
2/1/2009         21         9
3/1/2009         29         12
4/1/2009         30         11
5/1/2009         25         13
6/1/2009         29         16
7/1/2009         23         14
8/1/2009         21         10
9/1/2009         23         10
10/1/2009         24         12
11/1/2009         22         13
12/1/2009         28         17
13/1/2009         43         18


which I could cut & paste to Excel or a simple text file. However, with uFat I'm thinking that a data dump procedure may be a bit redundant, if I can simply pull out the SD card, plug it into my laptop & copy the file. One thing that I do need to keep in mind in this project is that it won't always necessarily be ME who services these cameras, so I need to keep their operation as user-friendly as possible...

Anyway, the idea with writing data to the SD card is that I may as well use the huge amount of capacity available & write data to it every minute. At this stage, I'm looking at recording data in the following manner:

Date, Time, Temp, VV.vv (Battery voltage), VV.vv (Solar Panel voltage)

where Date = DD/MM/YY (3 bytes)
          Time = HH:MM (2 bytes)
          Temp = degrees C (1 byte)
          VV.vv = voltages to 2 decimal places - (2 bytes each)

So, I'm left with 10 bytes of actual data going onto the SD card. However, in keeping with the spirit of simplicity mentioned above, I'm thinking that ideally the data written to the SD card would be in comma seperated value (.csv) format; that way we can easily import it into a spreadsheet for viewing & manipulation. This would add another 10 bytes or so for /'s, commas, decimal points etc.

My concern here is that 10 or 20 bytes obviously don't divide too well into 512; what happens when the buffer is full? Do I lose some data there? I'm not up to speed on exactly how sectors work, so forgive me if this is a stupid question... Could I make sector_buffer a 520-byte array (for example), write 512 bytes to SD card sector 0, then put bytes 513-520 back into the first 8 bytes of the array, to be written to sector 1 when the array is full again? Will a computer see this as one continuous file?

I think I'll leave it there for now - hopefully this answers most of your questions, and since this post is getting a bit(!) lengthy now I'll let you digest this lot & ask any more questions as required.

Cheers

JB
35  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: February 01, 2009, 05:05:37 pm
...rest of the code:
Code:
if (second == 0) {  // this code writes data to the sector_buffer every minute
   Serial.println("Writing data...");
   sector_buffer[WriteAddress] = day;
//   Serial.print(sector_buffer[WriteAddress]);
//   Serial.print(", day = ");
//   Serial.println(day);
   ++WriteAddress;
   sector_buffer[WriteAddress] = month;
//   Serial.print(sector_buffer[WriteAddress]);
//   Serial.print(", month = ");
//   Serial.println(month);
   ++WriteAddress;
   sector_buffer[WriteAddress] = (year-2000);
   ++WriteAddress;
   sector_buffer[WriteAddress] = hour;
   ++WriteAddress;
   sector_buffer[WriteAddress] = minute;
   ++WriteAddress;
   sector_buffer[WriteAddress] = byte(temperature);
   ++WriteAddress;
      span = 50;  unsigned int VBatt = 0;
      for (byte b = 0; b < span; b++) {        //loop to get average of 50 readings
        VBatt = VBatt + analogRead(1);
        }
      VBatt = VBatt / span;
   sector_buffer[WriteAddress] = byte(VBatt/79);
   ++WriteAddress;
   sector_buffer[WriteAddress] = byte(VBatt%79);
   ++WriteAddress;
// Note:
// The battery voltage on the divider ranges from 0 to 13.5 volts.
// The voltage on the arduino pin ranges from 0 to 5 volts using 2200- & 3900-ohm resistors.
// So an analogRead value of 1023 = 13 volts on the divider
// Therefore one bit is equal to 1024 divided by 13V = 78.769, rounded to 79.
      span = 50;  unsigned int VPanel = 0;
      for (byte P = 0; P < span; P++) {        //loop to get average of 50 readings
        VPanel = VPanel + analogRead(2);
        }
      VPanel = VPanel / span;
// *** NEED TO FIND OUT max V output of solar panel, divide 1024 by this, then replace 79 with the result. ***
   sector_buffer[WriteAddress] = byte(VPanel/79);
   ++WriteAddress;
   sector_buffer[WriteAddress] = byte(VPanel%79);
   ++WriteAddress;
   if (WriteAddress > 512) {
     SDWrite;
     ++sector;
     WriteAddress = 0;
   }
  }

  // check for any incoming serial input, then Dump Data only if 'd' is pressed
  if (Serial.available() > 0) {
    incomingByte = Serial.read();      
    if (incomingByte == 100) {
      DataDump();
    }
  }
 delay(1000);
}
There are a few redundunt bits & pieces, & lots of comments for my own benefit, but it shouldn't be real hard for the rest of you who are as new to this as I am to follow! smiley

Sorry, I've got to head out of the office now so this is a bit of a rushed response... If more info is required I'll check in again later today/tonight.

Thanks again,

JB.

-* EDIT *- After having a bit more time to digest your response I've made a couple of modifications to the code; it probably still has some gaping errors but it's semi-functional & allows for testing...
36  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: February 01, 2009, 05:02:57 pm
Hi Charlie,

Thanks for clarifying that for me, I think I get it now.

I think I can probably work with the 512 byte write each time... My program is monitoring a temperature reading and the voltages of a battery and a solar panel, in an enclosure housing a camera to take a few photos each day, producing 10 bytes of data per minute. I could probably drop this down to 8 bytes if writing 10 bytes/min causes problems with the 512 byte sector size, but I'd prefer not to.

Here's my code (over 2 posts):
Code:
#include <Wire.h>
#include <math.h>
#include <DateTime.h>
#include <avr/pgmspace.h>
#include <wprogram.h>
#include <microfat2.h>
#include <mmc.h>

#define ThermistorPIN 0            // input read pin for LM335 is Analog Pin 0
float temperature = 0;                 // variable which will be calculated in process
long val=0;                       // variable to store the value coming from the thermistor
float temp;

unsigned long sector = 0;
unsigned long byteSize;

unsigned long WriteAddress = 0;    // initial address for writing to the SD card.
byte value;                        // will be used...
int incomingByte = 0;                 // ...for incoming serial data
byte tempBytes[10];                // for sending data to the SD card
byte sector_buffer[512];           // for sending AND retrieving data to/from the SD card

int hour;
int minute;
int second;
int month;
int day_of_week;
int day;
int year;

char* dow[7] = {"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"};

float Thermistor(int RawADC) {  //  Inputs ADC Value from Thermistor and outputs Temperature in Celsius
 long Resistance;
 float Temp;                                      // Dual-Purpose variable to save space.
 Resistance=((10240000/RawADC) - 10000);          // Assuming a 10k Thermistor.  Calculation is actually: Resistance = (1024/ADC)
 Temp = log(Resistance);                          // Saving the Log(resistance) so not to calculate it 4 times later. "Temp" means "Temporary" on this line.
 Temp = 1 / (0.001129148 + (0.000234125 * Temp) + (0.0000000876741 * Temp * Temp * Temp));   // Now it means both "Temporary" and "Temperature"
 Temp = Temp - 273.15;  // Convert Kelvin to Celsius                                            Now it only means "Temperature"
 return Temp;           // Return the Temperature
}

void print_prompt_P(const char *data) {         // print prompts and messages from program flash to save SRAM
   while (pgm_read_byte(data) != 0x00)
     Serial.print(pgm_read_byte(data++));
}

void TakePhoto() {
  digitalWrite(3,HIGH); // first, switch on the fans to blow away dust or bugs...
  delay(10000);
  digitalWrite(3,LOW);
  delay(2000);
  digitalWrite(9,HIGH); // then send power to the camera
  delay(500);           // time for current to get to camera - not sure if this is needed...
  digitalWrite(6,HIGH); // trigger the solenoid for 100 msec
  delay(100);
  digitalWrite(6,LOW);
  delay(15000);         // wait 15 seconds for the camera to take a photo then shutdown
  digitalWrite(9,LOW);  // stop sending power to the camera.
}

void DataDump () {
  int DaysToDump = WriteAddress;
  Serial.println("");
  print_prompt_P(PSTR("          Data dump initiated: dumping "));
  Serial.print(DaysToDump/10);
  print_prompt_P(PSTR(" days of data.\n"));
    print_prompt_P(PSTR("Date,Time,Temp,VBatt,VPanel\n"));
      for(int p=0;p<DaysToDump;p=p+10) {
        sector_buffer[p]=digitalRead(13);
        Serial.print(sector_buffer[p]);

        Serial.print(analogRead(0));  // this bit dumps the date in 'dd/mm/yy' format...
          print_prompt_P(PSTR("/"));
//          ++f;
          Serial.print(analogRead(0));
          print_prompt_P(PSTR("/"));
          delay(50);
          Serial.print(analogRead(0));
          print_prompt_P(PSTR(","));
          Serial.print(analogRead(0));  // this bit prints the time, as ',HH:MM,'...
          print_prompt_P(PSTR(":"));
//          ++f;
          Serial.print(analogRead(0));
          print_prompt_P(PSTR(","));
//          ++f;
          delay(50);
          Serial.print(analogRead(0));  // this bit prints the temperature...
          print_prompt_P(PSTR(","));
//          ++f;
          delay(50);
          Serial.print(analogRead(0));  // this bit prints the Battery Voltage in 'VV.vv,' format...
          print_prompt_P(PSTR("."));
//          ++f;
          Serial.print(analogRead(0));
          print_prompt_P(PSTR(","));
//          ++f;
          delay(50);
          Serial.print(analogRead(0));  // this bit prints the Solar Panel Voltage in 'VV.vv,' format...
          print_prompt_P(PSTR("."));
//          ++f;
          Serial.print(analogRead(0));
//          ++f;
          delay(50);
        Serial.println();
      }
  WriteAddress = 0;
}

void error(const char* s)
{
  print_prompt_P(PSTR("Error: "));
  print_prompt_P(s);
  print_prompt_P(PSTR("<press reset>\n"));
  for( /* ever */ ; ; )
  {
    digitalWrite(13, (millis() / 250) & 1);
  }
}

void SDWrite() {
  // Pass in the sector-sized buffer we'll be using ourselves later.
  // uFat doesn't own it, it just needs to use it temporarily.
  // We also pass in the address of a function that is used to read
  // sectors from our device.
  //  
  if (!microfat2::initialize(sector_buffer, &mmc::readSectors))
  {
  error(PSTR("uFat init failed.\n"));
  }
  prog_char string_0[] PROGMEM = "DATA    BIN";

  if (microfat2::locateFileStart(PSTR("DATA    BIN"), sector, byteSize)) // check if the target file exists
  {
    if (byteSize >= 512)                                                 // check it's bigger than 512 bytes
    {
      if (RES_OK == mmc::readSectors(sector_buffer, sector, 1))          // checks a single-SECTOR read works OK
      {
          print_prompt_P(PSTR("Data read test SUCCESSFUL.\n"));
          delay(500);
        if (RES_OK == mmc::writeSectors(sector_buffer, sector, 1))       // checks a single-SECTOR write works OK
        {
          print_prompt_P(PSTR("Data write test SUCCESSFUL.\n"));
          delay(500);
        }
        else
        {
          print_prompt_P(PSTR("Failed to write updated data."));         // if the write fails, end here...?
        }
      }
      else
      {
        print_prompt_P(PSTR("Failed to read data.bin."));
      }
    }
    else
    {
      error(PSTR("Found data.bin, but it's too small."));
    }
  }
  else
  {
    print_prompt_P(PSTR("data.bin not present on card."));
  }
  Serial.println("Initialisation COMPLETE!!");
}

void setup() {

 pinMode(3,OUTPUT);       // FANS control - YELLOW wire
 digitalWrite(3,LOW);
 pinMode(6,OUTPUT);       // SOLENOID power control - WHITE wire
 digitalWrite(6,LOW);
 pinMode(9,OUTPUT);       // CAMERA power control - BLUE wire
 digitalWrite(9,LOW);
// pinMode(13,OUTPUT);
 Serial.begin(115200);
 Wire.begin();

// The rest of this setup() procedure is checking that the SD card is present & working.

  while(mmc::initialize() != RES_OK)
  {
    print_prompt_P(PSTR("SD init failed. retrying...\n"));
    delay(100);
  }
  SDWrite;
}

void loop() {

 // Below required to reset the register address to 0.
 Wire.beginTransmission(104); // transmit to device #104, the ds 1307
 Wire.send(0x00);
 Wire.endTransmission();      // stop transmitting

 Wire.requestFrom(104, 7);    // request 7 bytes from slave ds1307, we'll assume it'll send them all even though it doesn't have to
 second = Wire.receive();
 minute = Wire.receive();
 hour = Wire.receive();
 day_of_week=Wire.receive();
 day = Wire.receive();
 month = Wire.receive();
 year = Wire.receive();

 // Convert all the BCD values that might have "tens" to decimal.
 second = second/16 * 10 + second % 16;
 minute = minute/16 * 10 + minute % 16;
 hour = hour/16 * 10 + hour % 16;
 day = day/16 * 10 + day % 16;
 month = month/16 * 10 + month % 16;
 year = 2000 + year/16 * 10 + year % 16;
 
// Change times below to desired photo times. Copy & paste to add more photos per day.
// NOTE: for some reason, 8's & 9's cause an error, so don't use them on their own below;
// 18 & 19 work fine, but 08 & 09 do not.

 if (hour == 6) { if (minute == 00) { if (second == 00) { TakePhoto();}}}
 if (hour == 12) { if (minute == 00) { if (second == 00) { TakePhoto();}}}
 if (hour == 14) { if (minute == 00) { if (second == 00) { TakePhoto();}}}
 if (hour == 18) { if (minute == 00) { if (second == 00) { TakePhoto();}}}

 Serial.print(hour);
 print_prompt_P(PSTR(":"));
 if (minute < 10) { print_prompt_P(PSTR("0")); }
 Serial.print(minute);
 print_prompt_P(PSTR(":"));
 if (second < 10) { print_prompt_P(PSTR("0")); }
 Serial.print(second);
 print_prompt_P(PSTR(" on "));
 Serial.print(dow[day_of_week]);
 print_prompt_P(PSTR(", "));
 Serial.print(day);
 print_prompt_P(PSTR("/"));
 Serial.print(month);
 print_prompt_P(PSTR("/"));
 Serial.print(year);
 Serial.println();
  byte span = 20;  int aRead = 0;
  for (byte i = 0; i < span; i++) {        //loop to get average of 20 readings
    aRead = aRead + analogRead(ThermistorPIN);
    }
  aRead = aRead / span;
  temperature = Thermistor(aRead);

...continued next post...
37  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: January 29, 2009, 07:29:05 pm
Hi again,

After further reading (ie that uFat basically takes up the same space as the sd_raw instructions) I've headed down the uFat track. I've downloaded the uFat2 files from SirMorris's blog (http://groups.google.com/group/micro-hacker) and have integrated them into my program, although I have a few problems...

First, the sketch seems to 'interfere' with my DS1307 real time clock data. With the SD card connected as per the diagram at the beginning of this thread (and the clock connected to analog pins 4 & 5), the clock output is 'mixed up', e.g. the day comes through as the hour, year comes through as day of the month, etc. Leaving the hardware connected and loading up a sketch which ONLY displays output from the RTC, I get the correct output from the clock.

Secondly, I'm not too sure as to exactly how the reads/writes work. Am I correct in thinking that if (RES_OK == mmc::writeSectors(sector_buffer, sector, 1)) simply checks that a 1-byte write to sector (which=0) of sector_buffer works? Is it just the digitalWrite(13, (millis() / 1000) & 1) which writes sector_buffer out to the SD card? If that's correct, can anyone explain how this works for me? I'm very much learning as I go...

I'd like to be able to write data out to the SD card 10 bytes at a time (is this possible, or does the buffer have to be 512 bytes?), and be able to read them back & display them when prompted. Is this also possible, or am I only going to be able to check the output by checking the data.bin file on the PC?

Thanks again for everyone's thoughts...

JB
38  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: January 22, 2009, 05:11:37 pm
Hi Daniel,

That's certainly a nice-looking solution! Might be a bit of overkill for me though; using a 16Mb SD card that came with my camera, I worked out that if I store 10 bytes of data every minute, I've got over 2.5 YEARS worth of storage capacity...

Just for my curiosity, how would you communicate with this uMMC device? I'm guessing that you'd have to Serial.print() commands like "O 1 R /LOGS/JANUARY/JAN03.LOG{cr}" to it in it's own language?

Thanks!
39  Forum 2005-2010 (read only) / Exhibition / Re: SD card read/write with Arduino on: January 14, 2009, 06:31:46 am
Hi everyone,

Earlier, ducttapepro6 wrote:

Quote
Mr. Orange-

I had no trouble getting your code up and running in 15 minutes - happily logging, saving, and reading the input to analog1.

However, after hitting the arduino reset button, I found that the read function returns junk(from the SD card).

Shouldn't the SD card retain data after a power cycle? or, is it a function of the code that you so nicely supplied to get me started - and I just don't understand it, yet?
Along with Daniel, I'm hoping that someone can explain why the card starts up with the same junk data each time, if the first thing I do when the sketch runs is hit 'r'.

agent_orange says that it's like having a 1G (or whatever) EEPROM, which is IDEAL for my situation. My arduino will be sitting in a box, on top of a pole, powered by a solar panel for up to 3 months at a time. I have it successfully writing the date and max & min temperatures to the EEPROM, which I can read back out as desired. The EEPROM data is only overwritten after a new sketch is started. This is the functionality that I require from the SD card, but the supplied code appears to wipe the card each time the sketch is started. (I tried commenting out the if(!sd_raw_init()) but that didn't help).

agent_orange, thanks so much for supplying this work, it's fantastic & there's no WAY that I'd be able to work this out on my own. But if anyone can show me a way (even if it's a seperate sketch) to access the data on the card without destroying it, it would be GREATLY appreciated!!

I guess the other option would be if someone could recommend a Windows-based program which could access the data on the SD card?

Thanks for reading everyone!!
40  Forum 2005-2010 (read only) / Exhibition / Re: SD/MMC From the ground up on: March 03, 2009, 07:03:45 pm
@New Colin:

I have a sketch at http://docs.google.com/Doc?id=dqhc6fg_0gmk96kdd which uses the DS1307 with sirmorris's SD card software successfully. There's a LOT of code in there which will be totally irrelevant to you but I'm sure you can isolate the bits you need. If not, PM me & I'll cut the code down for you.

JB
Pages: 1 2 [3]