Dynamic Arrays

Simple question to start, is it possible to add to an array on the fly?

I'm doing a lot of programming in Processing to communicate with my Uno and found it quite easy for both with the language being so similar, however I feel that Processing is a little more advanced especially with having much more resources available.

I currently have one of my UNOs connected to the BMP180 module which is sending the results over to processing via the USB serial connection. processing then stores 24 hours of information using two arrays, the first being created every minute to get the average for the minute and the second holding every minute of the day from the minute array average. This is then plotted on a graph every 2 minutes to give a live feed of the last 24 hours.

My next stage is to use the SD Module that I have to create something similar but to write the data to an SD Card rather than over the USB. The reason for this is that I want to create a portable version and then take the card out at a later date to read the data.

The reason for a dynamic array on the portable version is that when I take the card out to fetch the data I want the Arduino to keep a log of data that can then be written when the card becomes available. To reduce the risk of corrupting the card I will have the write process happen once a minute and use pin13 to light up when the card is in use. Unless someone has a better option.

There's not a lot you can do with dynamic memory on a microcontroller without worrying about causing a crash. The problem is the very limited amount of RAM and no garbage collection. RAM gets fragmented to the point that the heap and stack crash even if it doesn't get all used up.

The thing to do is to keep a buffer handy that is the maximum size you'd need. Even if you were using dynamic allocation, you'd have to keep the block free in RAM anyway for it to grow into, so it's not like you're losing anything here. If you want to be able to keep up to 64 bytes, then allocate 64 bytes from the get-go. There's not going to be any way to get around having to have a maximum size when you're limited to 2KB of RAM.

Seems to me that you have well known requirements for the amount of data that you need to buffer:

  • Readings for every minute (say 1/sec = 60 readings), although a running average would prevent you needing to bffer all the readings.
  • At the end of a minute averages stored for 24 hours (60*24 = 1440).

Assuming that you have 1 byte per reading, this is already 1500 bytes of a 2k RAM space. You are not going to fit much more in an Uno…

I would suggest that you decide how much you will tolerate the SD card not being there (24 hrs? 48 hrs?) and then size the system appropriately.

As a data structure I would suggest a ring buffer (array with a get/put pointer) that will allow you to keep running, albeit with overwrite of the oldest value if you exceed the capacity of the arrays.

Thanks for the quick replies, this is what I was expecting.

At the moment the UNO is sending over something like "cc.cc,ff.ff,mmmm.mm,ii.ii"
The c being centigrade, f being Fahrenheit, m being pressure in my and finally the i being pressure in inches. These are all float numbers to 2 decimal places.

Data is currently sent to the PC roughly every 5 seconds.

I'm not looking at buffering a day of data just maybe an hour while I take the card out and copy the file. This way I can keep the UNO recording data and writing it back to file when it becomes available so at worst 12 records a minute for 60 minutes = 720 records.

Would I be best to have an array of the 4 floats or just create a string array to save the final string value?

Thanks for the quick replies, this is what I was expecting.

At the moment the UNO is sending over something like "cc.cc,ff.ff,mmmm.mm,ii.ii"
The c being centigrade, f being Fahrenheit, m being pressure in my and finally the i being pressure in inches. These are all float numbers to 2 decimal places.

First off, I would seriously consider reducing the amount of data that you send. Do you really need to record temperature in two different units? It seems like you would use one or the other, but if you do need both it is just a few lines of code to convert. I would just store one, if you know one you know the other. The same with pressure, why store in two units. Pressure is pressure, handle units when you do output.

Now let's think about the size of the data. You have numbers to two decimal places. How high do the temperatures go? If you multiplied by 100, would it still fit in an int? If so, you could store the number in a two byte int instead of a four byte float or five bytes for the four characters and a decimal if you do it in ascii. That would cut the amount of space in half or better right there.

Thanks again, yes cutting it down to what I actually need would make more sense.
Temp will be in C so it will easily fit in an int if multiplied and the pressure I work in my so that would go up to 103000 and should also be fine.

Thee may also be other data as I add on other units so keeping to a minimal is certainly better.

You could do something like this. But don’t forget to delete them once you’ve finished with them.

struct report
  float centigrade;
  float pressure;
  float somethingElse;

report *buffer[0];
int bufLength=0;

//Use this for your current values
report currentValues;

//save current values in buffer
void storeValues()
buffer [bufLength] = new report;
memcpy(buffer[bufLength], &currentValues, sizeof(report) );

I'd use INTs only. There is 65k different values to choose from.
If this is som kind of wheater station, pressure is in the range 950..1050 mBar -> no problem to get two decimals if needed.
Then if temp variation is within 25degC, u can get a decimal even in a byte of storage.

maybe the datasheet can give a useful hint here.

The use of pin 13, for the indicator LED, may be difficult, as its used for the SD-card interface.