Open Source Data Logger Project Using the Arduino?

I would even be prepared to start some sort of contest to kick start this project, throw in some prizes to generate some interest.

Anyone feel that this would be worthwhile?

I think a contest with prizes would discourage people from freely sharing their ideas and collaborating to solve problems. Maybe it's just me, but it seems antithetical to the sprit of the Arduino community.

I have a need for a data acquisition solution and I would be interested to know if it would fit into the scope of this data logger project.

I have a Wood Fired Brick Oven that I built in my back yard. It has several K Type Thermocouples (5 but other ovens could have up to 8) embedded in the mass to measure the temperature of the oven as it heats over several hours. I connected these thermocouples to my PC via a TC-08 Thermocouple Logger from PicoTech (See:TC-08 Thermocouple data logger | Pico Technology - I bought the older serial version) and ran a serial cable into the house ( about 50ft). This worked for a while but I the elements got to the logger and it stopped working. The latest version is USB and it's $500 without wireless capabilities. For a replacement I was thinking about a wireless version using Arduino and the XBee shield. To get 8 channels I guess it needs an multiplexer (or is it a demultiplexer?) module, and possibly a thermocouple A/D converter chip, to poll the thermocouples and communicate to the PC.

This real world example seems to highlight some of the common issues with data acquisition/logging applications:

  • Harsh Environments a the point of acquisition
  • Remoteness of the data acquisition point
  • Real Time data is required. In this case it needs to log the data to a real-time display. Logging to a local data store is not an option
  • Data visualization. It needs a software component to graph the results.

Another logging application I have in mind for Arduino is a GPS logger to geotag digital photos. This would involve logging time and location to a data store (SD for example) and then combining that information with the digital photos based on the timestamp on the photo.

If anyone is interested in a collaborative project for either of these applications. I would be interested. I do not think it needs a competition if there are members of the community that have a common goal. I see advantages for everyone by working on these types of applications under a common data logger/acquisition project. The results could be more modular.

Thanks, Robin

Maxim makes an SPI interfaced K type thermocouple interface, complete with reference junction, amplifier, ADC, etc. The Arduino can easily handle 8 of them via SPI. The arduino can also drive an LCD or LED display.

Bluetooth is another wireless option. The advantage over zigbee is that your computer may already have bluetooth. can't remember the max range off the to of my head, though...

-j

I did a bit of work on this and got an SD card working (reading/writing) no problem which makes it very easy (and cheap!) to store large amounts of data. Using even a small SD card u could capture data at reasonably high sample rates for a long time. I also added a max1307 real time clock to add timestamps. I did a bit of work on an arduino shield for the circuit with all of this on it but haven't sent it off yet.

To get 8 channels I guess it needs an multiplexer
You can add an 4051 which will give you 8 channels on one analog input and use 3 digital lines.

Real Time data is required/Data visualization
This is actually easier than storing it in a storage device. Just write the results you get to serial. On the pc side read them in and graph them (processing can do this) other options could be python or VBA. There is also a few programs around to do this. A quick google found this one http://www.live-graph.org/

I'll post up my circuits and code once I have it working ok and everything is cleaned up and its not too embarrassing :stuck_out_tongue:

everything is cleaned up and its not too embarrassing

I would like to start a campaign on these forums for people to start releasing code when it's still at the "embarassing" stage. The number of times a thread ends with a post saying "I'll release it when it's tidied/not embarrassing/perfect" and no further code seems disturbingly high.

Embrace your code's imperfections in the face of its utility, release now!

In a month's time untidy, embarrassing code is still going to be a lot more useful to others than no code--it also probably won't impact your future employment/dating opportunities too much. :slight_smile:

--Phil.

P.S. This isn't aimed specifically at you, this thread just happened to be active as the thought occurred to me. :slight_smile:

everything is cleaned up and its not too embarrassing

I would like to start a campaign on these forums for people to start releasing code when it's still at the "embarassing" stage. The number of times a thread ends with a post saying "I'll release it when it's tidied/not embarrassing/perfect" and no further code seems disturbingly high.

Embrace your code's imperfections in the face of its utility, release now!

In a month's time untidy, embarrassing code is still going to be a lot more useful to others than no code--it also probably won't impact your future employment/dating opportunities too much. :slight_smile:

I agree! I think few of us are real professionals here and we're all here to learn, even a brain dump of imperfect information thrown back into the cloud is better than nothing at all, and we can all benefit from dissecting and optimizing what you've got so far. :slight_smile:

It's great that you brought this up. I'm actually working on using an Arduino to store data off to a USB thumb drive. I'm logging from one USB port and saving to the thumb drive. Just need to get the closing of files working correctly.

I'm a co-op student working for a company right now and am highly interested in working on this project. I do have some questions about the actual operation of the data logger though.

  1. How good is the resolution? I know it's 10 bit, but what is the max value for the analog input? Our main need here is thermocouples and 1 mV can be 20 degrees difference or more.
  2. How could we get more inputs? I know you can get more boards or more A/D converters and then put them on the digital ins, I was just wondering if there was an easier way to do this?

My main goal is to get as many thermocouple inputs as possible. Any code I write, no matter how "dirty" would be posted back.

Sorry if most of the questions have been answered elsewhere, I'm just now getting into the arduino board community. Most of my experience is with Linux based microcontrollers.

Thanks.
Ryan
Tenn. Tech U.

Hi Ryan, by default the arduino uses 5 volts for the ADC reference. As I recall from the datasheet, you can use an internal reference of 1.1 volt giving about 1mv resolution. The guaranteed accuracy is plus/minus 2 bits.

There are six analog inputs and you can add an external multiplexer if you need more.

Has anybody taken on this project? I have a few ideas for a modular system. I think better resualts could be gained by using standalone ADC chips, combined with a modular system, you could just plug in the modules you need!

hey Ryan,

Maxim has a couple of thermocouple interfaces (MAX6674 MAX6675) that would attach to the Arduino via SPI. Since SPI is a bus, requiring a separate chip select for each device, you could easily put a dozen or so on an Arduino as-is. With an IO expander IC you could add even more.

I've got these on my list to play with, but haven't had time yet.

-j

(TTU class of '90)

Has anybody taken on this project? I have a few ideas for a modular system. I think better resualts could be gained by using standalone ADC chips, combined with a modular system, you could just plug in the modules you need!

I'd be interested in making one if it was a competition or something.
I just dont have any need for a data logger myself though.

kg4wsv I2C would be a better candidate.
Up to 128 sensors with only two wires.

kg4wsv I2C would be a better candidate.
Up to 128 sensors with only two wires.

SPI has two distinct advantages: SPI is much much faster that I2C (25MHz vs 400kHz, IIRC), and, in this case, the thermocouple interface ICs are only available in SPI. :slight_smile:

On the number of nodes on the bus issue, I have noticed many ICs that have fixed addresses, or allow only a few bits of the address to be set by the designer, so practically this limit is usually much lower than 127. For example, the DS1631 that was discussed on the forum recently has 3 pins (3 bits = 8 possible addresses) available.

-j

SPI can hit 25mhz theoretically but the Arduino doesnt go that fast anyway.

Apparantly you can get I2C hubs which allow more.

I have been looking for I2C thermocouple ICs and I just can't seem to find one. Like K said, only SPI. Since the MAX chips are more expensive than an analog mux, I think I'm going to use one MAX6675 and then just mux all my inputs through it. It's easy and dirty but should work. I have a few other ideas I'm still messing around with before I start ordering samples.

This method also leaves open my analog inputs since we may need to take voltage/current readings at some point.

By the way, speed is not a priority for me since most of our measurements are 2 hours apart, but I will try to make it as fast as possible.

Hi everyone,

I am a biologist from a wildlife research group in Australia. I have next-to-no experience with electronics, but I am wondering whether a novice like me could learn to build a simple GPS datalogger based on Arduino boards (preferably the Arduino Mini) for use in wildlife tracking? I would appreciate comments from the community here as to whether this is something a newbie like me could attempt, and whether anyone might be willing to assist? Obviously any progress made will be posted here for others to use. Look forward to comments.

Thanks

Yes a newbie could do it.

All you need is the processor (Mini), a GPS module and some form of memory (I2C EEPROM is easiest).
Whack them together and you have a basic datalogger.

If you want to monitor birds or small animals, I'd leave it to the experts though.
It wouldnt be the smallest datalogger in existence. :wink:

Hi everyone,

I am a biologist from a wildlife research group in Australia. I have next-to-no experience with electronics, but I am wondering whether a novice like me could learn to build a simple GPS datalogger based on Arduino boards (preferably the Arduino Mini) for use in wildlife tracking? I would appreciate comments from the community here as to whether this is something a newbie like me could attempt, and whether anyone might be willing to assist? Obviously any progress made will be posted here for others to use. Look forward to comments.

Thanks

I think that kind of depends on what you're a novice at, but it's not outside the realm of possibility at all. Start small. Get an Arduino and start playing with it. Then find a suitable GPS receiver and start playing with that. Figure out how to get the position and other data out of it that you want and send it back to the computer over serial. Then find a way to log your data to some kind of storage device (probably the hardest thing to do right now, from what I've seen) and tie it all together. I think if you break it down into smaller pieces, you'll be ok. I'd recommend you find a good tutorial on C and C++ if you're not already conversant in it, however.

The easy part is connecting the GPS, there is some information on connecting a GPS in the [u]Arduino playground[/u] , this one is a good place to start: Arduino Playground - GPS

The challenge will packaging the complete system to be small and robust with a suitable power source, a practical means to get the data out, and an antenna layout that has sufficient visibility to GPS satellite data when attached to wildlife.

You may want to start a new thread where you can say a little more about your requirements and encourage contributions on various solutions. I would definitely think its worthwhile pursuing further.

Hi Cheater, blalor and mem,

thanks for the tips and encouragement. I guess it's a case of diving in and giving it a go. A separate post might be a great idea if there is enough interest.

We've been trialling the latest GPS dataloggers from Sparkfun, and as much as they're great little units, the "sleep" mode has a 1.5mA draw, that really takes its toll on battery life. Getting the sleep draw down to the lowest possible level is a major goal of a new logger. A recent post on the Arduino playground discusses sleep draw: (Arduino Playground - ArduinoSleepCode).

From what we can tell, a major portion of active draw in the SF logger seems to be related to NMEA sentences being written to the SD card. Fast(er) fix acquisition and a less intensive approach to writing/storing fix information is another goal.

As you guys have pointed out, size is also a limitation. We can probably work with a final board measuring ~50mm x 50mm and ~30mm high (although the smaller the better). The Sparkfun board measures 38.1mm x 44.5mm x22.9mm), which is close to ideal. Patch antennas are a bit of drama to work with and ultimately we'd like to mount the GPS logger complete with antenna and battery on the underside of collars. At the moment, either the antenna or both the logger and the antenna have to be top mounted, which makes fitting in teh field a drama.

Thanks very much for the replies guys, and I look forward to more discussion.