Real time clock module help for design project

Projects General Guidance
1

I am trying to create a device to record and keep track of a hypothetical individual's weight, so that I can see the trend/changes over time. I'm using the real time clock module to keep track of time and a load sensor to measure the weight (converting resistance readings to corresponding weight). I got both component up and running, but I am having trouble getting and recording the time from the RTC when I want to record a weight measurement.

So for example, I want to take a weight measurement every 10 minutes for 5 hours and have all the data points stored in a central location. These data points should contain the weight measurement as well as the time it was taken, so that I can graph it and see the trend.

Of course, this is hypothetical. I am simulating 10 minutes as a month in real time.

I attached my code for the RTC and load sensor (they are two separate codes). Any help as to how to record the time each measurement is taken would be very helpful. Thank you in advance!!

Project_RTC_setup.ino (3.25 KB)

Final_Project.ino (544 Bytes)

2

If you are going to immediately insert the weight measurement in a MySQL database you could use TIMESTAMP. RTCs are often unreliable, so this method avoids that issue all together.

3

lafontas:
RTCs are often unreliable...

It's odd that I hadn't noticed that.

4

Had my industrial logistics hat on when I made that comment. The need for battery changes and clock drift make RTCs a poor production solution, but an adequate hobbyist tool. Cheers

5

lafontas:
Had my industrial logistics hat on when I made that comment. The need for battery changes and clock drift make RTCs a poor production solution, but an adequate hobbyist tool. Cheers

So it would be a bad idea, say, to put one into every personal computer built in the last couple decades or so? They are hard to notice, because the battery generally lasts several times longer than the computers.

Depends on requirements. If an atomic clock is needed, yah an RTC would be a poor solution. Multiple semiconductor manufacturers make dozens, probably hundreds of different RTC models. They do not do this to satisfy the hobbyist market, which is hardly even a drop in the bucket for them.

6

I think you might be missing the fact that the clock software is for setting not running. It is almost the same as what I use but is a confused version. It is better explained here.

http://bildr.org/2011/03/ds1307-arduino/

The DS1307 is not super-accurate but is perfectly suited for what you want. Ultimately, you only want the calendar.

It appears you need to combine the two programmes which, in principle, is simply a matter of bringing the preamble, setup and loop stuff from one and combining it appropriately in the other. You need to consider how this is being used. The whole procedure may be better in the setup and the there is no loop i.e. one-shot on power-up. As it stands you may get a different reading every two seconds with no means of determining which one to use.

The other thing to work out is is this thing a stand alone device, or an interface for a PC? If the former, you will probably need on-board recording but don't ask Iafontas what to do about the clock. If the latter, you probably don't need the clock. You just need to get away from the serial monitor, as most proper terminal programmes can derive timestamping locally.

7

Hey Jack, I appreciate the dialogue. I'm relatively new to the Arduino having only used them for a couple of years. I have however spent my career as a technologist/manager (I have the grey hair to prove it :). I built a somewhat large Arduino network on my farm and like many others I included RTCs in the design. I had a very poor result with these inexpensive clocks and in the end despite my best efforts I had to abandon them in favor of a using a network time model. I take your point about the RTCs in PCs, however I would point out that most of these devices are regularly adjusted by network time servers which in effect recognizes their shortcomings and, I have had to allocate valuable resources to replace batteries in hundreds of PCs at a substantial cost.

In ordering various RTCs I received a number of units with dead batteries and so I had the further aggravation of ordering LIR2032 cells at $6 - $10 per battery. Arguing the status quo for example how PCs are built, doesn't mean they got it right despite the products success. Again, thank you for the dialogue, I appreciate your passion. Cheers. Steve

8

OP,

Where is your problematic combined code?! We don't care about working code that much. Give us your broken code.

Jack and lafontas,

I agree with both of you. I have more than dozens units out in remote fields for all seasons and endures cold winters. The reason some cheap RTC loses time is the crystal. The crystal that you most often see, as a little metal tube with two leads, are not temperature regulated and thus suffer from temperature variation. Combined with the fact that many of them were made for watches, the intended operating temperature, at a human's wrist, is many degrees away from actual operating temperatures, say by 30-60 Deg C or F or whatever you use.

I don't have my solid state physics book at hand, but I trust wiki. "2 minutes per day for temperature 10 deg C above or below room temperature".

My personal experience: some have problems with bad broken crystals, some had dead batteries and need constant change, possibly poor quality IC, and the rest just work happily and be off by some minutes when I check them.

If you can afford crystals contained in temperature regulated ovens, go with those. Otherwise, get some accurate ones that uses temperature probes to compensate for temperature effects.

9

lafontas:
Hey Jack, I appreciate the dialogue. I'm relatively new to the Arduino having only used them for a couple of years. I have however spent my career as a technologist/manager (I have the grey hair to prove it :). I built a somewhat large Arduino network on my farm and like many others I included RTCs in the design. I had a very poor result with these inexpensive clocks and in the end despite my best efforts I had to abandon them in favor of a using a network time model. I take your point about the RTCs in PCs, however I would point out that most of these devices are regularly adjusted by network time servers which in effect recognizes their shortcomings and, I have had to allocate valuable resources to replace batteries in hundreds of PCs at a substantial cost.

In ordering various RTCs I received a number of units with dead batteries and so I had the further aggravation of ordering LIR2032 cells at $6 - $10 per battery. Arguing the status quo for example how PCs are built, doesn't mean they got it right despite the products success. Again, thank you for the dialogue, I appreciate your passion. Cheers. Steve

Hi Steve,

Sounds like we may have somewhat similar backgrounds. I have a fair amount of experience with several RTCs, from Maxim Integrated and from Microchip, and all have performed exactly as advertised by their datasheets. So I'd have to ask where the disappointing units came from. There have been a lot of threads on the forum here about problems with inexpensive RTC modules, usually from eBay or other sketchy sellers. These are not representative, there is some real junk out there. Batteries that are DOA are a sure sign of a disreputable seller and I'd definitely cross them off my list. I only use components from top-shelf distributors like Mouser and Digi-Key. I offer an RTC module on Tindie.com and have sold a fair number of them with only good reviews. I've had several repeat customers including an order for over a dozen that I believe wound up in another product.

Maxim DS1307 is my least favorite, it's low on features and as @liudr points out, its accuracy is determined strictly by the crystal, typically ±20ppm, which in my book is marginal accuracy. Maxim's DS3231/32 are much better due to a built in temperature-compensated crystal. These can and do deliver ±2ppm.

The Microchip MCP79411/12 which I sell on Tindie are interesting RTCs. Loads of features, one of which is a calibration register. Even though these also use a ±20ppm crystal (I don't use the metal cylinder type, but rather a SMT crystal from Citizen), they can be calibrated very closely. I had one running for nearly a year (at fairly constant household temperatures) that was off by less than 0.5ppm.

As for Lithium coin cells, I am definitely in the wrong business :wink: I use CR2016s in my RTCs. A package of two will go for $5-$6 at the local SprawlMart, yet I can order ten from Mouser (Panasonic brand) for around $0.30 each. Go figure. One of these should last 5-7 years in an RTC as standby current specs are typically given in nanoamps.

I too have an Arduino/XBee network. The data concentrator node interfaces to the web, and also syncs its clock periodically with an NTP server. The sensor nodes in turn update their clocks from the concentrator. Some nodes have RTC chips and some just rely on the MCU's system clock, for which I usually use a ±20ppm or ±30ppm crystal.

So that's the view from here. As long as expectations are aligned with the specs in the datasheets (including the crystal's datasheet), then if there's still a disconnect, I would not look to the RTC first, but rather to circuit design, counterfeit parts, etc.

I too appreciate the dialog, hope to see you around the forum! Best regards ... jc

PS: Well we have totally hijacked the OP's thread at this point, apologies for that. OP, if you're still out there, give us an update, let us know what's going on.

10

Hey pkrs, as Jack mentioned we hijacked your post with an engineering debate on real time clocks. In your original post you talked about "how to record the time." I looked at both sketches you provided and by "record" did you mean display as serial output or maybe data log to an SD card?

Jack and Liudr,

I appreciate your good thoughts on RTCs. Liudr I liked what you did with the Phi-shield and bought a couple of your boards from Dip Micro and corresponded with you briefly on your blog. Jack I'm going to head on over to Tindie and have a look at your RTC offering as sounds like you have a quality product.

Cheers,

11

pkrs,
I would suggest using some higher level libraries to make dealing with "time" easier.
One thing that I wold encourage you to think about is exactly how to track time.
There are two ways to track time:

  • as a local datetime which is the date & time in the local timezone
  • as a fixed timezone usually based on a epoch, which is essentially just a integer number
    that can be converted back to any local date time based on knowing the local timezone.

This can be important if the data is shared between multiple entities/people that
may span timezones.
If this is the case, then storing the the local datetime is a bad choice.
When using UTC time that is based on a fixed epoch (the way unix tracks time)
then you can always figure the local time by simply knowing what timezone you are in.
If the datetime was stored as a local time, then in order to figure out the local time
you have to know not only the local timezone but the timezone where the datetime
was originally capture. Even then it is a fairly painful calculation to convert a local datetime
to another local datetime. In unix format you simply add/subtract a simple constant
from the integer datetime number. That is why using the unix way is much better.

There are some higher level libraries that avoid having to deal with the RTC and
the timezone complexities to allow the application to simply deal datetime - in either local time or UTC.

Have a look at the Time library for dealing with local time:
http://playground.arduino.cc/Code/Time

Jack has some nice libraries:
For handling timezones: GitHub - JChristensen/Timezone: Arduino library to facilitate time zone conversions and automatic daylight saving (summer) time adjustments.
For DS3231/2 RTC: GitHub - JChristensen/DS3232RTC: Arduino Library for Maxim Integrated DS3232 and DS3231 Real-Time Clocks

By using the Time and Timezone libraries you can deal with datetime
at a higher level than talking directly to the RTC chip.
This can allow you easily change the underlying low level time tracking between different
RTC chips should the need arise.

--- bill

12

lafontas,

Great! I'm happy to hear that you have some phi-shields. Dipmicro no longer carries my kits. Now I'm selling on my blog and inmojo.com marketplace. Shipping charge to Canada has doubled since 2013. It's a lot harder to sell like I did before, to all parts of the world.

13

I just started learning about RTC's the other day. I found one the other day on the motherboard for a non-working NightOwl DVR. I pulled it off along with the coin battery holder and crystal and put it on a small piece of perfboard and got it working. The one I have is PT7C4337WE. Don't know about the accuracy, but it's working okay for my needs. :slight_smile:

14

The problem I'm having is exporting the month and year data as well as the weight measurement into excel/EEPROM where it can be stored and recorded. Any suggestions?

15

pkrs:
The problem I'm having is exporting the month and year data as well as the weight measurement into excel/EEPROM where it can be stored and recorded. Any suggestions?

If it were me, I'd use the Time library as bperrybap suggested. It's actually three libraries in one download, there is also a basic DS1307RTC library that works well.

Unless of course you're in it for the chase, and that's fine too. In that case, please confirm whether the code in the original post is still current, and describe what you're getting for output.

16

pkrs,
Something to think about is that is the ideal form for the data for the processor
and storage inside the AVR eeprom, for a human, and for excel are not the same.
i.e. the format of the timestamp data you store in eeprom can and probably should be different
from the other two.

That is one of the benefits of using an epoch based time value. It is much easier for
the processor and for storage. Then convert it back to other formats when a human wants
to see it or when exporting it to something like excel.

If you use the Time and timezone libraries you can keep the format of the timestamps
in an epoch format which is a simple 32 bit integer which is easy to store in the eeprom,
then use libraries to convert it to whatever format you need for any timezone later when needed.

It is a change of mindset, in that you no longer store a timestamp as a
local time & date in the typical human desired format,
but rather an epoch datetime which is a representation of
when the event occured which is independent of locality or any timezone.

For exporting to excel, the AVR could convert it back to a local date/time human readable format
but this can also be done in the spreadsheet very easily.
If you google search you can find many examples & tutorials on how to do this conversion in excel.

--- bill

17

pkrs:
The problem I'm having is exporting the month and year data as well as the weight measurement into excel/EEPROM where it can be stored and recorded. Any suggestions?

You still don't say how you want to go about this. As I said in my previous, if you have Arduino connected to a PC, you don't need the clock and you can do what you want with PLX-DAQ, It is a freebie macro for Excel which effectively makes it into a live terminal, i.e. you send direct to Excel. Excel gets the time off the PC and all you do is tell it where to put it.

Here are some notes on it. For Arduino, it is just a matter of sending like you do to the serial monitor plus the right Excel commands.

http://homepages.ihug.com.au/~npyner/Arduino/GUIDE_2PLX.pdf

If you feel you still need an on-board clock, i.e. Arduino is portable, that in the bildr link enables you to pick out the date, and just use that. I don't think an EEPROM merits your attention. It is neither fish nor fowl, and isn't worth the effort, while backing up to a local SD card is easy and well-covered in the IDE examples.

Sending the data is essentially the same as sending it to the serial monitor

  myFile = SD.open(filename, FILE_WRITE);//<<<<<<<<<<<<< OPEN
  myFile.print(hour);
  myFile.print(":");
  myFile.print(minute);
  myFile.print(":");
  myFile.print(second);
18

I would do epoch and human time on sd card or serial port, unless you want very dense data points.

19

liudr:
I would do epoch and human time on sd card or serial port, unless you want very dense data points.

Are you suggesting storing multiple timestamp formats?

I have a general rule that I encourage others to follow when storing data:
Never store data that is easily derived or calculated from other stored data.

I would suggest to decide up front if epoch datetime, or local date & time
are to be used for storing timestamps and stick with only one.

There are benefits and drawbacks to either form.

Pick one format for storage and convert to others, if needed/necessary.

--- bill

20

Yes, it is easy to inspect data without excel. With how cheap an sd card is, a 4gb card probably provides a lifetime supply of storage for the project. I don't see any point in saving space or time.

For OP, close file after each data point to prevent loss and file corruption. Suggest you to use sdfat library, not sd.