How long to store data?

We don't know the purpose of data retention(nor, apparently, does the OP), so commenting on either "how long", or "how much" is pretty much useless.

So how many counters are you going to use? Didn't get it

Didn't you mean go to 1?

Exactly. Hence my post #38

Going to 1 would restart the initial value. Going to 2, data will be stored as "increased/decreased" (by a known amount... degree, tenth of a degree??) form, taking up bits, not in a whole value taking up bytes. Time interval is known.

This is pseudocode... my guess is one... for "sameTemp" then all the data is "offset from initial value at a known interval" Greatest range of temperature (worldwide) in a day is 57c. Extreme temperature ranges where humans using computers live is 20c.

I agree. Furthermore, the OP does not seem to have sufficiently clear ideas about its objectives, so it is difficult to help more than that.
In my opinion, if even the goals of the project are not clear, it is useless to continue giving advice. Thus, I'm done here.

Sometimes people just like to see graphical representations of data. I used my online energy fee statistics to graph my annual usage. As useless as that was, a graph of yearly differences helped me locate an issue causing a large difference... and the graph enlightened those who needed to know.

So a step "saving the data" is missing in your explanation. Here either we increment, either we go back to 2 (indefinitly)

So at one moment you need to save the value of this counter or it will be lost. Same remark as above

Giving steps as you did is good, but not when steps are missing. It could just mess with what the OP (or the others) has in mind. I think you should consider editing your post

However, the OP needs to come back with more details if he needs more help

We have no flipping idea of purpose, yet we diddle around the edges with conflicting suggestions and solutions. Timewasting, indeed.
As with @docdoc, I'm out. "Fill yer boots, matey!"

Pseudo code.

During "reading" value is compared against previous offset. No value is stored. Only offset.

Questions and answers fill in the blanks.

I do.

Purpose of what?
Why do we even need to know?

Surely it matters given the title of the topic

Thinking out loud:

Quickly going through this thread I had the following thoughts:
(datasheet)
The DHT21 measures temperature -40..80 C (+- 1C)
The DHT21 measures relative humidity 0..100% (+- 3%)
Time between two measurements >= 1.7 seconds.

Given the accuracy one need just 1 byte for T and one byte for H.
This allows 0.5C and 0.5% values if one wants to, but given the accuracy not meaningful.
For T one needs an offset of -40.
==> 2 bytes per measurement (in fact some values are not used YET)
So storage looks like [T][H][T][H][T][H][T][H][T][H][T][H].... etc

If one has a number of measurements one can check if one can apply run length compression. If multiple measurements are the same it would be possible to compress them. This can be encoded as [Run length flag][length][T][H] or short [R][L][T][H].
Given this length of 4 bytes. the run length compression is more efficient if there are 3 consecutive measurements that are the same.

As we do not know the actual measurements it is impossible to calculate the savings in storage but they could be substantial. (which is an indication you measure too often).


A second way of saving storage could be that one makes measurements every 2 seconds and add these to a circular buffer (size 30).
Once a minute the average is calculated and stored in a second circular buffer (size 60) for the last hour.
Once per per 30 minutes the average of the minutes is calculated and stored in a 3rd buffer (so 48 samples per day).

So 3 buffers, one high frequency last minute buffer, one medium frequency last hour buffer and one low frequency half hour buffer. Only the last need to be stored in EEPROM.

Given you have 128 KB storage == 65535 measurements \which are half an hour apart. 65535 x half hour = 1300++ days so more than 3 years.

By "playing" with the buffer sizes and the frequency of averaging you optimize this idea to your needs. E.g. if the long term only needs one average per hour you go to 7+ years of storage.

Purpose of what?

Hello all, Sorry I got called into work way to early. I have been thinking about what everyone had said. Being on limited storage is not fun when you have to store a lot and you have to figure out ways to make it all fit.

I came up with maybe a simple way instead of logging everything for a 1to7 days worth of logs. I think for me the most simple way to go is just log the current one and display it. I will create a mysql database to log more if I need to that way I know I'm not running out of space. For me this is the better way to go. I know there is cautions in doing this.

What happens if you lose connection to the database. Yeah same thing if If the external storage goes bad. I lose everything too. I thought about the pros and cons. For this way I think it is a better way for me. I'm exploring all options and Yes I will be doing sampling as well before storing it to the database. I'm looking at that more closely because I have never done that before. I will update this post in a couple of days.

Another pro about putting it into a mysql database is that I can timestamp it as well. If I was to do it locally to the arduino itself It would of taking more space up. this way I can have it all go to the database.

What about the mean of n+2? 1-3, 3-5, 5-7 etc.

If we don't know the purpose for which the data is being stored then how can we provide advice on how long to store it for ?

We do. Time v. Temp, plotted.

How frequently will the temperature be recorded ?
Over what period will the temperature be recorded ?
With what precision will the temperature be recorded ?
Is temperature the only data to be recorded ?
How is the data to be displayed ?

Only with those facts known it will be possible to suggest storage methods and I don't think that we are there yet

The answers are in the first few posts, far above the dozen foot stomping posts. Size limit, duration, granularity.