Pages: 1 2 3 [4] 5 6 7   Go Down
Author Topic: Open Source Data Logger Project Using the Arduino?  (Read 36610 times)
0 Members and 1 Guest are viewing this topic.
0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Robin,

I am exactly in this type of project, trying to make a rugged versatile data logger. After a couple of poor attempts, I focussed on the SD card as a memory device. My strong advise is: SPI! Not much litterature to read on that (except official doc), but not too tough neither. And so much room! I store data in a 512 bytes buffer which matches exactly the size of the SD sectors and leaves you a good additional 512 bytes for the code on the arduino. I placed all the strings in PROGMEM. Even my spare 16Mb SD looks huuuuuuuge!

My data logger can be programmed from a computer and parameters are not volatile. So that you can unplug from usb and plug to DC supply and control runs from one single button (start/stop/clear all runs smiley-razz). Runs (and their related headers, containing acquisition parameters) are stored sequencially using a tricky feature which prevents loosing the data in case of unexpected power failure. Rough specifications are: records interval from 10 ms to 1 hour (watch out the rollover of timer), averages from 1 to 16, channels from 1 to 6, programmable nbr of records (or infinite if 0). There is a list and print command for reading headers or data from each run.
Further development could be: LCD display, high speed DAC, programmable ranging, etc.
I do not plan to tease you just for fun, but I say: yes, it is possible! I will probably publish it soon smiley-wink
Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Just a couple of words to keep you updated. Version 19 is under test. This version includes runs listing, runs printing (selectable), deletion, online help, test mode, upload mode, real time clock (bloody timer roll over  >smiley-sad ). But! I cannot shorten the cycle time below 40ms, due writing cycle time (I was looking for 10ms). May be I am a little too ambitious for waht is not a scope, right?
Next development steps are: self probe type recognition, and bulletproof microcode.
Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Self recognition of probe is running fine (8 # types can be defined). I managed to shorten the write delay to 30ms which is better but does not fully statisfy me. The articles that I found on the net are very encouraging: SD itself says that single block writing is prefered in the case of 512 bytes buffer size, multiple blocks works faster thanks to the fewest "waits for iddle after writting". The prototype fits in a 12x11x4 cm small box. On the other hand, I am reaching the limits of Arduino's memory! From this perspective, we are back to the early 80ies  smiley-grin

I am a little bit surprised by the lack of reactions to quite an active topic smiley-wink
« Last Edit: October 01, 2008, 10:10:11 am by joker » Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 2
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello Joker
I only just joined the discussion, please do not dispair!
Your project looks very promising.
Currently I am looking for a Arduino logger to monitor the energy used by my electric geyser. I need to simply measure the time it is used over a 1-2 week period, at the same time i also would like to monitor the geyser as well as the water inlet temperatur.
How far are you from publishing?
Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Solar toy,

What is an electric geyser  ? :o What is the temperature range you are interested in? In the begining, I will restrict the publication of the whole stuff to the members of my car club. We will see then. But I do not mind releasing portions of the micro code. Thus my questions  smiley-wink
Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Beraking news: I cut down the writing time to 15ms thanks to, Oh yeah, I know this was trivial... a fast SD card...
The flash scan allows now one kind of scope acquisition (100µs resolution), 512 measurements for one channel, 256... for twice more, etc.
Now comes the question of uploading to a wireless network. Using the 300 bytes left in Flash memory...  smiley-razz
Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 2
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello Joker
A geyser is the thing in your attic that usally stores 150l of hot water.
Without it showering is only half the fun.
I do not need a complex logger. One measurement every minute is enough. As long as I have 2 off 0-5V analog inputs and 2 digital inputs I can help myself further to connect the sensors. The unit needs to operate stand alone, its a bit difficult to bring the PC into the attic  smiley-wink
Any help is much appreciated. I use a Diecimila.
Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I see! Do not ask me why, the trivia name for it in French is balloon (hot water balloon to be accurate) :o
I am in the process of fine tuning the prototype and its micro-code. You will have to use a shield for the SD card: you may consider Adafruit's one (blank GPS shield) or make your own from the Adafruit's proto shield and hack a 1$ USB SD card reader. Then you might not need all the functions that I designed (100 bytes left in Flash...). I will consider publishing a lighweight version.  8-)
Kind regards
Didier
Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 29
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Joker,

Joyeux Noel!

I read with interest about your datalogger.  For many people, a generic datalogger (or datalogger library) could form the basis of their latest project.

SD memory is definitely the cheapest option, and being able to buffer 512 bytes in the Arduino RAM and then write a block to the SD card is probably the most efficient use of memory and power saving.

For many,  a few analogue inputs, which can then be interfaced to a variety of sensors (light, temperature, current, voltage etc) plus some digital pulse-counter channels would satisfy their requirements.

Are you close to publishing your code? I would be most interested.

I have an Arduino and Ethernet Shield and I am serving the output from some analogue sensors to the internet via Pachube - see it here:

http://www.pachube.com/feeds/1257


G

Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Guinnessimo,

Thanks paying attention to my project. Where am I? Well, ahead from my early plans. What was a curiosity to me is slowly turning to a real project... I showed the prototype to the CEO of my group  8-), and he is very enthusiastic...  smiley

So that the publication of my works depends on the production of the project which slightly moved towards the concept of an "intelligent sensor" for environmental monitoring. I will let you know  smiley-wink
Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 29
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Joker,

Thanks for your quick reply.

I am interested in using the Arduino to monitor home energy consumption with the aim of allowing users to save energy and save money.

Additionally, the Arduino with an ethernet shield is one of the cheapest solutions (< 30 Euros)  to how you can serve real time data to the internet.

With many users sharing their home energy data, it then becomes a competition to use the least energy, and thus lower their domestic carbon footprint.

The Arduino could also be used to measure the output from renewable energy sources such as solar water heating, windpower and photovoltaic.  

If many people shared their renewable energy generation data via the internet, it could be used to build up national statistics about the quantity and distribution of renewable energy installations.  

If we are expected to reach 20% renewables by 2020, then we must find an effective way of monitoring our progress and logging the installations.

Already some people have solar water heating controllers, and electricity meters that output data - and perhaps the <30 Euro Arduino is a good low-cost way to publish this information to the internet.



G
Logged

0
Offline Offline
Newbie
*
Karma: 1
Posts: 41
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Guinnessimo,

I have a strong concern for these energy saving questions. If I had the time... I would investigate the question of the "intelligent" house too. I feel a little sorry not to disclose my works so far.

About the Ethernet connection: As you know, France is part of the "old continent" and most houses are not fitted with networks or what so ever... Mine was built... 2 siecles ago! Thus my concern for wireless...
Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 29
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Joker,

By 2020 the EU will have smart meters for electricity and gas.  Italy already has started fitting them now.

The utility companies are very slow to adopt new technology - they do not want to spend money unless they have to - or are forced by EU Law!

In the meantime, there are a few low cost energy monitors such as Electrisave/Cent-a-Meter, CurrentCost, Wattson and a few new ones just coming to the market.  Some of these have data output which can connect to a PC or serial logging device.

Someone has worked on the Electrisave wireless protocol so that it can be decoded by a PC or Arduino - using just a very low cost 433MHz receiver (taken from wireless doorbell).

For many people, just to see approximately how much energy they are using, and how much it costs, is enough to make them save.  The accuracy of the energy monitors is not too good - perhaps +/- 10%, but that is close enough for the average home user.

For those without ethernet or broadband internet there are several ways in which a logger can get data to a server via a plain old telephone line.  It is possible to implement the DTMF dial and simple modem functions into an ATmega168.

I hope that you will soon be in a position to share your project ideas more.


Bon Annee a Vous!


G

Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 29
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Joker, List,

Here is an excellent description of a home-built datalogger based around ATmega and SD card.

http://www.turtlesarehere.com/html/data_logger.html

It includes the code and the schematics/layout in Eagle format.

The hardware is simple to adapt from an Arduino.

The "Captain's Universe" is a good source for interfacing SD card to ATmega.

http://www.captain.at/electronic-atmega-sd-card.php


Happy New Year,




G

Logged

Salem, Oregon
Offline Offline
Sr. Member
****
Karma: 7
Posts: 310
Friends don't let friends use Strings
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Nice link G...

I've been working on my own device which boils down to a data logger with a gps and an XBee attached, so I've been following this thread with some interest as well.

I've also found these links to other GPS loggers to be interesting:

http://hackedgadgets.com/2008/11/30/gps-data-logger-using-an-sd-card/

http://www.gedanken.demon.co.uk/gps-sd-logger/ (the completed unit picture seems to be missing now)

At any rate, here's some more food for thought.

Brad.
Logged

Pages: 1 2 3 [4] 5 6 7   Go Up
Jump to: