Coloumb counting data logging

I trying to track power usage on different circuits. I walk into things not seeing how complicated it gets.

I'm measuring current and voltage so I can get the power by V * I / Time. Then accumulate the power and time.

I'd like to be able to view this data in a chart over an hourly period, 24 hour and weekly.

There is the complicated part.

How do I do the data logging?

Some questions:

  • How fast should the power readings be to have acceptable accuracy? (500ms? 2 sec?)
  • It's on an esp32. I multiply the float value current and voltage by 1000 so integer maths can be used. Do I need to be concerned about the accumulated values overflowing a 64bit int?
  • I could use a high power raspberry pi with influxdb to dump the times series data in but it's on a boat so I want to limit power consumption. I believe a pi zero is pretty weak for the influxdb task. No cloud connection either.
  • Do I keep a bunch of arrays the different time intervals? (60 floats for one hour, 48/day) Updating becomes messy.

Nothing beats the Sparkfun OpenLog. It is a tiny, independent module that writes everything you Serial.print() to a file on an SD card. CSV files are convenient and can be read directly into a spreadsheet on a laptop.

As for the other questions, it is up to you to define "acceptable accuracy". Experiments will reveal all.

2 Likes

It will make the program more complex but you might consider only logging data if the data has changed by +/- . It will significantly reduce the logged data but it makes the data analysis more complex.

PMFJI
I saw the module for the first time, it's interesting.
But...
Looking at the schematic and firmware source code, It's using the ATmega328P with 3.3V at 16MHz.
That is not guaranteed condition, it doesn't look like a recommended module...

I agree about 16 MHz on 3.3V, but I've been using one for years and have never had a problem. Sparkfun claims they have never seen a problem related to that clock choice either.

1 Like

Yeah. :blush:
In my experiments, this generation of AVRs behaves at 16MHz when greater than between 2.9V and 3.1V on most lots. (well, I've only tried about ~40 uCs)
It can be said that no problem will occur when if 3.3V is supplied stably.

However, I insist on many beginners to "obey the component rating" at this forums.
So I keep in mind identically I don't take overloading.
This is my ego. :wink:

That OpenLog idea could work. Even if I just use it as a buffer and wake a pi or similar up once a minute to process the data into a database.

Off to go see if it's feasible to have a device waking, processing then sleeping. It probably doesn't need to be once a minute processing unless I want to see the data, in that case it can just run real time while on.

That opens up higher powered devices without worrying about overall consumption.

I don't have one to try right now, but If I recall the Arduino IDE combined 3.3 with 8Mhz and 5.0v with 16Mhz.

Is it simply a matter of programming the 3.3V board and telling the IDE it is a 5v board?

Also thinking more....
I assume you have to reload the boot loader? If I had gone this far I think I would use optiboot bootloader.

You can make your own openLog from that on an ordinary 5V uno or mini, I did and used an cheap SD card module off ebay you just compile and upload there source code in it.

https://www.ebay.co.uk/itm/322502815052?hash=item4b16aa6d4c:g:9B4AAOSwPIhaNuch

This has the 5V to 3.3V conversion already built in. So you don't have to worry about working on 5V side of things. I then just sent the serial data to it as you would normally with the original openlog.

I made the serial data format so I could use the EasyLog USB software to view the graphs and I logged data every 2-5 minutes when I logged some data on battery charging and discharging test on some big batteries and this gave good viewing results.

looking at the frequency of data collection is really based on what your application is.

a solar panel can get sun, clouds, sun in a few seconds.
or you can record watts per hour.

if you break down the watts, you have an overall watts per hour, so for that data you can record it.
you can also look at peak output and time. and have a second data file

I had played with degree days. this is the outside air temperature based on how far from 60F.
each degree would be logged.
for the hour, average temperature, highest and lowest.
then how many minutes were at 59 deg, 58deg, 57 deg, etc
I was logging all temps just once per hour.

temperature and solar panels are rather slow.

what is your process ?

There's solar, wind gen, alternator then the load circuits (fridge, autopilot, etc). Wind gen super variable so higher sampling is better.

The process was using reimann sums to get estimates of power. It's basically just Current x Volts x Time delta. It can get more accurate using trapezoidal measurements and such but lowering the time interval is also good.

I just wonder how writing to the SD card and reading it to another device can work. It's all over serial so pretty slow going.

On the link to sparkfun, the product listed speed.
It also linked to another device that was much faster.

Are you wanting to log a 10 bit value of a sine wave? 60Hz times 10 so 6KHz?
Times 3-phases?

@JohnRob: The IDE doesn't check on the voltage (to my knowledge), so just tell it it is programming a 5V (16 MHz) board.

I modified the OpenLog code to add time stamps, so I know the overall process works. But that was several years ago.