Best approach to time-keeping for battery-powered project

I'm prototyping a device for a wildlife research application (always working on a shoestring budget!). In electronics I'm out of my comfort zone, which of course is why I'm using Arduino. My device has to record events, with an interpretable date and time, + data from sensors, and it must run off batteries. To conserve battery power, I intend the device to be in sleep mode most of the time. Of course I must also be able to download the data somehow (lots of options there, perhaps the simplest is to connect another Arduino device, which I carry about from unit to unit). Because I will need many of the independent field units (100+) they must be cheap, so after doing my development on an Arduino Uno I am aiming towards a stripped down version of chip + minimal componentry.

My question concerns how best to approach the date/time keeping aspect?

In my prototype to date I have used an SD shield to record data, but I figure that I can use the flash memory on the chip itself (I think the capacity is enough), and dispense with the SD shield. But the SD shield has an additional battery and uses this to maintain its own system time. If I dispense with this shield or equivalent clock circuitry, I must depend on the Atmel chip's internal clock. If I understand things correctly, the internal clock is not brilliantly accurate?

Now, one of my sensors uses UART to communicate with the chip, though this could be changed to Wiegand. From what I understand, UART requires more precise timing than the internal clock delivers. I can't discover whether this is also true for Wiegand. So, am I bound to use add-in clock circuitry? Or if not, how is best to initialise and then interpret the internal clock?

Would be extremely grateful for pointers.

Simple answer would be to use a good RTC device. Ones with DS3231 chips are very good where ones with DS1307 are a bit hit and miss on accuracy. They have a backup battery so can maintain time for year(s) without external power. The DS3231 also has a temperature sensor and 2 alarms that generate an interrupt so if timings between readings are in the minutes/hours you could use the alarm to wake the arduino from sleep.

Storing your data in RAM is not a good idea as it will all be lost if power is interrupted. Cheap option would probably be to use an SD card then you can just swap it out for a empty card and download at leasure.

EDIT: Just noticed this that has the DS3231 and a 32K flash memory. Maybe ideal for storing data.

I think you're looking for EEPROM (or an external storage device such as an SD card) rather than flash storage.

There is an Arduino clone designed for use as a data logger which may be a good platform for you to use - it is small and inexpensive and has an integral SD card slot.

The simplest way to get long term timestamping would be to use an RTC. If the cost is prohibitive you could consider using a cruder solution based on the Arduino's free-running internal clock plus a LSR to give you approximate dawn/dusk times and some post-processing to guess at the long term clock drift.

PeterH, which Arduino clone were you thinking of? There seem to be many, and I find the choices pretty confusing.

Riva, I wasn’t proposing to store to RAM, for the reason you stated. But surely flash memory does remain when you power down? Isn’t an SD card a type of flash memory? Apologies if I am in a stupidly confused state still. Anyway, that little clock/temp/eeprom board looks very interesting - thanks for that.

EEPROM is built into many of the Arduinos. It is memory that you can write to and read from using functions. So you could store a limited amount of data in EEPROM. It depends on the chip how much if any EEPROM you have (Due doesn't have any for instance, Uno has 1KB, Mega 2560 has 4KB, etc.). There is a limit of the number of writes you can do the EEPROM and while it is pretty large, you probably don't want to update it every time the loop function gets called. The only way to get to EEPROM is via the Arduino, so you might need a toggle switch that says whether to dump the current EEPROM bytes or not when powering on. Alternatively, you could download a different sketch to read the EEPROM values: http://arduino.cc/en/Reference/EEPROM

SD/micro SD cards have much more space. You use functions that create files and write to them. You do have to worry about whether the file you created is more than 2 gigabytes, due to it using the FAT16/FAT32 filesystem. With a SD card, you can remove the card, and move it to your computer (or cell phone, etc.): http://arduino.cc/en/Reference/SD/

In addition to the RTC devices, if you are already using a GPS chip for figuring out the location, the GPS chips provide much more accurate time than the RTC devices do (at least when they are active and can see the birds in the sky).

Some of the Arm boards (teensy 3.0, divx, etc.) that use Arduino-like programming have a RTC builtin if you add the appropriate crystal and a coin battery for the RTC.

Seeeduino Stalker combines the time chip, SD card, XBee footprint, and a solar charge input.

Chagrin: Seeeduino Stalker combines the time chip, SD card, XBee footprint, and a solar charge input.

If you don't need the xbee or solar charging, Adafruit has a somewhat cheaper shield that just has the SD card and RTC: https://www.adafruit.com/product/1141.

snusmumriken: PeterH, which Arduino clone were you thinking of? There seem to be many, and I find the choices pretty confusing.

I was thinking of the Hobbytronics ardulog board, although there are other similar products from other suppliers.

Now, one of my sensors uses UART to communicate with the chip, though this could be changed to Wiegand. From what I understand, UART requires more precise timing than the internal clock delivers. I can’t discover whether this is also true for Wiegand. So, am I bound to use add-in clock circuitry?

I routinely use TTL serial @9600 from an ATmega328P-PU running @8 MHz on its internal RC clock. From a reading on Wikipedia and some timing info posted here:
RFID Access Zone. It appears that Wiegand protocol will have an advantage only in long physical cabling runs and the required pull-up resistors (220 Ohms) for the 2 data lines will add to the overall battery draw. If the cabling run is short, I would think the serial interface would be preferred.

Using the uC in RC mode will definitely open your project to frequency shifts from ambient temperature changes. The price of an 8MHz crystal and two associated load caps will add minimally to your project when considering you are purchasing in multiple 100’s quantity.

Ray

snusmumriken: Riva, I wasn't proposing to store to RAM, for the reason you stated. But surely flash memory does remain when you power down? Isn't an SD card a type of flash memory? Apologies if I am in a stupidly confused state still. Anyway, that little clock/temp/eeprom board looks very interesting - thanks for that.

My bad, I thought you had said RAM. AFAIK a program cannot write data to it's own flash memory. Only the bootloader can do that.

Wow, some very valuable pointers here. Thanks so much, everyone, for your time.