Hi all,
A person by the name of Nick did an article in the playground called "Writing data to USB Memory Stick"
i am interested in writing to a USB memory stick. one of the things that i want to do is figure out is how to write a date and time stamp for each of the samples i take. my application is a field data logger. is there some way i can do this. also i don't understand some of the set up that he did like tying the rts cts together.
I don't need to read to the USB stick really only log data to it. does any body have any sample code related to time and date stamps for data logging.
also i'm an interested in contacting nick off line for some questions
thanks
Jeff O'Brien
To get a reliable timestamp you would need to hook up a real time clock IC to the Arduino board.
This thread http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1191209057/15 discusses that
You could also, with a little more day-to-day hassle for yourself, in exchange for a simpler set up, use the Arduino's millis function...
Reset the board when you put in in place to do the logging, make a note of the date/time (Ink- on- paper, in a notebook!)
Have the data logging Arduino log (to the flash device) the result of millis at the time of each reading.
Once you have your data in your "big" computer, it would be easy enough to "translate" from millis to date/times.
If it's outdoors, you could also use a GPS. A bit of overkill, but might be useful anyway.
Just be aware that the millis() function will roll over and start from zero after aproximately 9 hours
Hi
I originally added some stuff about using rtc to the playground stuff to create a proper timestamop for the file - it is easy to change that to log it to the stick with any reaqdings, I'm around if you have any other questions.
Oh - I connected the rts to cts so that I didn't have to do handshaking. This means that if you send a lot of data quickly it may not write it properly as the usb memory stick gets written to a lot slower than you could send data to it from an arduino. If you are sending a small amount of data not very frequently then it will probably be ok - you will need to test your plan and see if the results are satisfactory.
Hi
I originally added some stuff about using rtc to the playground stuff to create a proper timestamop for the file - it is easy to change that to log it to the stick with any reaqdings, I'm around if you have any other questions.
Thanks nick
I will be looking into this later tonight
here is a question...
what are the pros and con of usb memory stick vs. sd card?
Keep in mind this will be outdoors.
i haven't bought the VDIP1 yet and want to make the best decision
Jeff
Short answer on the technical front is - I don't know - I wanted something that was as simple to understand as possible for the people using it. They are more familiar with, so less worried about, usb memory sticks, and can get at the results on any computer without needing a card reader. There are other posts on the forum that may help decide which is technically better.
what are the pros and con of usb memory stick vs. sd card?
The SD card is easier to wire up but its harder to interface.
That memory stick chip makes it easy to use the memory stick but its harder to wire up.
Hi Nick
a few pretty basic questions.
I have read your article probably twenty times. i just want to see if i am getting it.
so i buy vdip1 or vdrive from mouser. i hook up 5v and gnd and digital pins 0 and 1 to the transmit and receive of the vdip1. i then hook rts cts from vdip1 to each other. i will be sending 9 bytes and a carrige return once per second. Is this slow enough and small enough amount of data for the rts cts hook up?
When i call the WRF command does it just automatically append to the file?
Does it format the drive everytime it boots up or can i put a settings file on there and have it leave it alone?
here is where i need some help.
If i also add the dallas 1307 rtc how is connected and communicated to while the vdip1 is connected and being communicated to? don't we only have the pins 0 and 1 on the arduino for serial comm? or am i missing something.
thanks
Jeff
A few comments on just the....
Don't we only have the pins 0 and 1 on the arduino for serial comm?
...part of the above...
Well... yes and no.
For "industrial strength", easy, serial comms, yes... use pins 0 and 1
And yes, trying to talk to several devices all interconnected to those pins will cause problems.
If you only want to send something from the Arduino over a serial line, then the provisions within the....
....software serial library are great.
However, trying to read FROM something external, using SoftwareSerial isn't such a rosy picture...
a) There's no async data buffer
b) If you invoke a Serial.read, the system will "hang" inside the call until some serial data arrives.
the dallas connects using different pins as it uses i2c to communicate with the arduino.
I wouldn't be worried about 9 bytes once a second (but you will need to test it).
It doesn't format the drive each time you start it up - but while you are developing if you send bogus codes to it or fail to close a file you have opened you can corrupt the file, so only work on a 'spare' memory stock rather than one with anything important on it.
I think WRF opens ready to append - check the documentation pdf for the vinc1 (link at the top of the playground page).
If it is battery powered I would consider opening to write, writing the values and then closing it on each pass so that if the power fails at some point you are more likely to have a readable file as it will be closed for a fair amount of the loop time.
Thanks a lot nick, i'll be ordering my vdip1 today.
i'm sure i'll have questions once i get the gear, so i'll try to keep this thread going.
thanks again
Jeff O'Brien
I've been thinking about writing my 10 bytes of data to the usb memory stick once per second for potentially a year.
if i haven't messed up the math, there are 31,536,000 seconds in a year multiply that time 10 bytes per sample and that will create a 308mb file approximately. does anyone have any guess or real life experience with opening and appending to a file of that size on a USB memory stick with a micro controller. i'm guessing that this is going to be a problem. i guess one option might be to break it up into multiple files that hold about a months worth of samples.
Let me know what you all think of this.
thanks again
Jeff
I don't think working with a file that big should be a problem. It's up to the vdip1 to manage the operations around opening and appending to the file. The Arduino code won't really have any concept of what's going on on the thumb drive. 300MB isn't really that big. 
That being said, keeping a monthly (or weekly or daily) file will spread your risk out, should something happen to the file.
Depending on where the final kit will be logging stuff I would be concerned that there is no way to check that the file is ok till the year is over - and then you may have nothing. You would be gambling on no power issues in a year, that your circuit is robost enough to survive where it is for a year etc.....
I would split it into files - either one for each day or week.
If you are going to be checking how its doing then i would also write in some way indicate when to arduino is writing/or not and a way (switch) to briefly suspend writing to file so that you could remove and replace the memory stick without it all going pear shaped so you can check intermediate results etc....
HTH
Hi Nick,
I got my VDrive2 today and will begin work on this tonight.
i have a question about debugging. below is some text from your tutorial
start quote
I'm using an Arduino mini for my app., and have a usb to serial board for programming it. While I am debugging it I have the serial out of the Arduino going to the serial in of the vdip, and the serial out from the vdip going to an additional serial in lead that I am monitoring with Z term on the mac. This means I can send messages to the Arduino from the serial monitor in Arduino, and see the response I get in Z term.
end quote
what is the additional serial in lead that you are monitoring with Z term. or maybe i should ask how do you set it up?
also i am using an Arduino decimillia in case it matters
if it does matter, might you suggest a setup for me to be able to debug with the decimillia?
Thanks
Jeff
Slightly off topic... but maybe another "answer" to the sort of thing that SOME data->USB experimenters are trying to solve.
Crude? Perhaps. Robust? I think so!....
One "answer" to collecting data is to hook the Arduino to a line printer with a good ribbon.
The data can be printed, ink-on-paper, as it is collected.
Subsequently, the hardcopy can be transformed to machine readable data via a scanner and OCR software.
My biggest doubt about this approach is whether OCR software can do a good job on dot matrix printing, which is what most of us are likely to have in the junk room for a line printer for this job. Of course, printing in an enlarged font is one answer, but that adds hassles. (If anyone has had success with extensive OCR data capture from dot matrix printed stuff, please send me a private message, and if interest arises, I'll add the information to the relevant thread?)
If someone wants to "run with" this alternative, please start a new thread?
Of course, this solution requires a little more space and effort... but it will be a rare time that already-collected data gets lost by a storage medium corruption!! Ideal for, say, access control records when you want to check "what happened" only once in a blue moon. Not so good, perhaps, for something like weather data.... although, of course, the printer could be drawing a graph at the same time as it was capturing data for future analysis.