Open Source Data Logger Project Using the Arduino?

We need additional electronic anyway to handle all of our inputs. So maybe we are designing a shield for that.

As it looks the arduino mega fits pretty well.

Hi

New to the forum, new to Arduino

As the Science director in a UK High School, what we need is not another data logger that is either too expensive or too complex -- but a data acquisition tool that "just works".

A basic box that had say:

a) 4 channels
b) RTC
c) Sample frequency from ms to daily
d) Basic LCD feedback
e) Minimal memory (cheapest possible) USB or SD Card

As far as schools are concerned, the logger is just a "black box" and it's the sensors that you plug in that make them useful.

A version that I decribed above should only cost $40 (ish) to make -- If such a device existed, they whould sell bucket loads in the UK.

This is the basis of my interest and research into the Arduino.

Cheers
G

Glen,

Have you seen the Logomatic II?

No....

x8 Analog / Digital inputs -- wrap it into a neat box with some control circuitry -- might be just the job.

Still -- I'll press on with the Arduino -- makes an interesting project.

Good feedback - thanks
G

Glen, please check out the Duino644 board, Arduino-compatible:

Hey Guys

i've been working on an online data logging system built on google's app engine and i've pretty much decided to go open source with it. I'd really appriciate some feedback from Arduino users and perhaps even some help.

I've worked with data historians for 12 years and i wanted to write my own.

The system i built allows you do log time series data over a rest web service and your data is stored on google's infrastrucutre. I wrote a .net sdk and an way to view/record data on an android phone.

i'm release the code soon under the MIT licence.

I posted some code in my blog on how to wire devices into the web service - it works great, you can see values change in spread sheets and diagrams.

Anyway - i would really appriciate any feedback - you can contact me at bsautner@ (google's free email server)

thanks!

Ben

Sorry, the web site for my data logging system is http://www.nimbits.com

The source code will be posted on Google Code:

http://code.google.com/p/nimbits/

I spent 6 years working on this thing, so i hope it can be a good starting point for a cloud based data logger.

Getting back to issues involved in developing an open source data logger based on the arduino:

It seems to me that the development of a shield to handle analog signal conditioning is a key step in making the data logger a broadly appicablel tool. The ten bit resolution of the Arduino A/D converter (ADC) is not an issue as long the signal from sensors is relatively linear and varies so that the voltage change covers a large fraction of the 0 to 5 volt scale. However, in many cases, the range of sensor outputs will vary over a much narrower range, and programmable gain amplifiers will be needed to adjust the dynamic range of voltages that are presented to the ADC. (e.g. resolution of the ADC will not be adequate if sensor output voltages range from 0 to 0.5 V). Also, as many sensor's output varies logarithmically or exponentially with respect to the stimulus, an exponential or logarithmic conversion will be needed to prevent loss of resolution at the low end of sensor input. Based on my limited knowledge of OP amps, it seems that a shield to handle this kind of basic signal conditioning might not be too expensive to make. Of course this leaves out some more specialized signal conditioning needs (e.g. cold junction compensation for thermocouples), but if the shield could handle simple variable gain amplification and logarithmic transformation, it would allow the Arduino's 10 bit ADC to serve adequately for many, many data collection situations.

T

Hi, I am new in this forum. I am very interested to invest my time and some money in a data-logger development based on Arduino. I think that the problem must be divided into some steps: how to interface sensors to arduino input, that can be solved with easy electronic circuits;
how to get arduino writing data on a SD card or USB pen drive;
how to get enough power autonomy for long data logging (months).

In fact I think to use this sort of device to monitor slow environmental phenomena (i.e. wine cellar temperature :)).

I can recommend the GHI uALFAT Module for data storing. Its working pretty well.

Ben,

had a look at your nimbits stuff - I like the idea of using google as a data store.

Do you have some sample arduino code that you would use ? I'd like to test the idea with some temperature logging...

Thanks
Siegfried

instead of trying to make a "one size fits unknown" datalogger
I think the first thing would be to list all the sensor types likely to be hooked to this datalogger and from that determine what input conditionning will be required at minimum for more kinds of sensors
so at least it's known which input conditionning facilities should be on a standard datalogger and which type can be left off a add-on adapter board

also someone should do an inventory of all open source datalogger project that can be found and enumarate their features and scan them for "good ideas" to incorporate in this design

datalogger are often needed but they come in so many brands and model and industries because the people who measure temperatures over weeks don't know about the people who measure fuel/oxygen ratios during minutes and they don't know about the people who measure viscosity over months but they can all benefits from a single common datalogging core with just add-on for their specific application

datalogging for most is an expensive and R&D intensive affair that has no re-use value that's why people pay 500/1000/2000$ for a datalogger that have specs all over the place and there's a reason for that , there's big money to be made in keeping each project unique and personnalised and impossible to compare/shop around for

and lastly the most important of all, this project needs a good name that clearly indicate what it does to people who need it yet don't know what it is yet and it should be easy to find so that the next time someone needs to log the humidity of their bathroom over the next year they don't re-invent the wheel again !!!

in the event of the Internet connection going down is there anyway to save the log updates and sync them when the connection is restored?

Siegfried, i think you found these already - but here is the blog posting on how to connet arduino to nimbits.

http://nimbits.blogspot.com/2010/07/data-in-connect-arduino-to-nimbits-data.html

tublar - check out www.nimbits.com, there is a lot of client code both .net and java - it'd be pretty easy to make a buffer in your code on the lan if the internet connection dropped. I may just add that to the code library since it's a pretty basic need.

  • Ben

Hi, I'm very new and a scientist interested in dataloggers.

I'm mainly interested in one that can work by sending data to a computer as it analyzes a signal. Does something like that exist? This wouldn't require battery power or very much data storage...

I am thinking about creating a specific website which will act as a central resource for open source data logger projects, allowing the less technical to communicate with us with there project idea's. Additionally, It might also work as a place for developers to share code between each other and exchange idea's.

I had plans on competitions, prizes, etc to generate interest and give some incentive for everyone to contribute, I have had both positive and negative comments on implementing this.

What do you you all think? Good or bad idea?

I noticed that this thread has been viewed over 10,000 times!! It would appear that there is a lot of interest out there.

I think it a brilliant idea! The views of this topic is nearly hitting 50,000! :o
So was the central depositry site for data logger projects created yet? ;D

Guys,

I will do this / collaborate on this. I already manage an open source data logger that connects arduino to google's cloud computing infrastrucure. I have the time, resources and interest to keep something like this going. I have the code to put up a full text search web site with user authentication

Please contact me if there already is a site that's been started if you just need my help.

I own the domain opensourceloggers.com which i will donate to this cause

Here is an under construction web site for Posting and searching for open source data logger projects. I'll get the data structures security set up over the weekend and will keep posting progress to this thread:

http://www.opensourceloggers.com/

It'd be very helpful to hear about what sort of catagories data loggers should fall under - such as Arduino, SQL, JDO etc - maybe the language, licences type etc. Posting a description, repository location...anything else?

Hi kurofune,
"sending data to a computer" over the USB port is easy on Arduino.

It can be captured on the computer using Processing, and plotted or saved.

I am very new to this field and I am building Data logger for a racing kart or a car. I think there should be a category for this type of loggers.
Although, they are similar to any other type of loggers, there are specific aspects of them being exposed to a very hostile environment i.e Rain/water/oil, vibration, EMI. I have faced this problems and finding ways to tackle them, as I am sure many others too. For example SD card slot doesn't take vibration very well and the card tend to move and eventually disconnects, which is not very healthy for logger in a middle of a busy logging session ;D
It would be nice to share other ideas about the racing data loggers with fellow motorracing enthusiasts.