Pushing values to array

Hi,

I'm working on a project where I need to keep track of previous values, like 'rolling' data over the last few measurements. I do this by pushing a value into an array and at the same time discarding the oldest value.

This is both:
-- Working code for anyone how might be looking for it
-- An ask for advice on how to improve it, make it nicer.

The code works but there might be a nicer, more elegant way?

Thanks for reading!

#include <Streaming.h>                                  // for serial output

const int numberOfElements = 10;                        // Declare as const for setting size of historicValues[]
int historicValues[numberOfElements];


void setup()
{
  Serial.begin(115200);
}

void loop()
{
  pushAtStart(random(0, 9));
  //pushAtEnd(random(0, 9));
  delay(1500);
}

void pushAtStart(int newValue)
{
  for(int i = numberOfElements - 1; i > 0; i--)         // Shift current values to the right starting at the last element
  {                                                     // element 10 (at index 9) gets the value of element 9, 9 gets the value of 8, 8 the val of 7, etc.
    historicValues[i] = historicValues[i-1];
  }

  historicValues[0] = newValue;                         // Place the new value at index 0

  for(int i = 0; i < numberOfElements; i++)             // Print the array to screen
  {
    Serial << historicValues[i] << " ";
  }
  Serial << endl;
}

/*
Output from pushAtStart should look like:
(entries shifting right in each new line)
8 6 8 0 1 8 8 6 8 4 
0 8 6 8 0 1 8 8 6 8 
2 0 8 6 8 0 1 8 8 6 
7 2 0 8 6 8 0 1 8 8 
7 7 2 0 8 6 8 0 1 8 
*/


void pushAtEnd(int newValue)
{
  for(int i = 0; i < numberOfElements - 1; i++)         // Shift current values to the left starting at the first element
  {
    historicValues[i] = historicValues[i + 1];          // Element 1 (at index 0) gets the value of element 2, 2 gets the value of 3, 3 the val of 4, etc.
  }

  historicValues[numberOfElements - 1] = newValue;      // Place the new value at the last index

  for(int i = 0; i < numberOfElements; i++)             // Print the array to screen
  {
    Serial << historicValues[i] << " ";
  }
  Serial << endl;
}

/*
Output from pushAtEnd should look like:
(entries shifting left in each new line)
6 5 5 4 0 5 8 3 4 3 
5 5 4 0 5 8 3 4 3 4 
5 4 0 5 8 3 4 3 4 5 
4 0 5 8 3 4 3 4 5 0 
0 5 8 3 4 3 4 5 0 2 
*/

A more elegant method rather than moving all the elements of the array is to leave them where they are and change the value of the index to the array to point to the start of the values

1 Like

To add to what @UKHeliBob said:

[edit]

Here is a possible implementation using an integer array.

template <size_t n> void push(int (&arr)[n], int const value) {
  static size_t index = 0;

  arr[index] = value;
  index = (index + 1) % n;
}

template <size_t n> int pop(int (&arr)[n]) {
  static size_t index = 0;

  int result = arr[index];
  index = (index + 1) % n;
  return result;
}

Usage:

int val[3];
push(val, 1);
push(val, 2);
push(val, 4);
Serial.println(pop(val));  // Prints "1".
Serial.println(pop(val));  // Prints "2".
Serial.println(pop(val));  // Prints "4".

[edit2]

A small variation on this theme seem to be in line with your needs:

template <size_t n> void roll(int (&arr)[n]) {
  static size_t index = 0;

  for (size_t i = 0; i < n; i++) {
    Serial.print(arr[(index + i) % n]);
    Serial.print(' ');
  }
  Serial.println();

  index = (index + 1) % n;
}

Usage:

int val[3];
push(val, 1);
push(val, 2);
push(val, 4);
roll(val);  // Prints "1 2 4".
push(val, 8);
roll(val);  // Prints "2 4 8".
push(val, 16);
roll(val);  // Prints "4 8 16".

Note that this is only useful if the order of the elements is important, otherwise val can be used directly.

@UKHeliBob: I think I understand what your saying. That would be a nice solution if I was to, for example, calculate a rolling average over the historic values. I'l keep it in mind. But do let me know if I'm wrong ok.

In my case the order of the element is of importance.

Say I want to pushAtStart 3,2,6,4 into historicValues[3]

In my code that would result in:
[0,0,0]
[3,0,0]
[2,3,0]
[6,2,3]
[4,6,2]

If I understand correctly you suggestion would result in:
[0,0,0]
[3,0,0]
[3,2,0]
[3,2,6]
[4,2,6]

I'm gonna have to give it some more thought OK?

@jfjlaros: Looks like I got some reading to do, thanks for the link!

Your interpretation of my suggestion looks OK and you can read out the values in the same order that your example gives by starting at the index position of the head and moving through the values, wrapping round the index value when required

Don't forget that using the index to array method you can either increment the index or decrement it to suit yourself. As you know where head of the list of values is you can read them in both directions too but it often does not matter as in the case of a rolling average

Thanks for the reply.
I really have to give this concept some thought.

The Wikipedia article @jfjlaros links and the internet at large have some implementations that are simpler, using just regular C kinda code.

push and pop are also odd names for a circular buffer. The article uses put and get.

Slightly more sophisticated yet still clear and easy examples will worry and take care of putting too many things this overwriting things yet to be getted, so to speak.

But yeah circular buffers, tons of examples to be googled up.

a7
.

The circular buffer is a good idea.
Yet, there is a more simple method that might work for you.
It seems you want (in the end) a rolling average.
Often a rolling average is replaced by a weighted rolling average to give the last value more weight and increase responsiveness.
If that is the case, you might like:

int av(int new_val) {
   static int old_av = 0;
   int new_av = (new_val + 15*old_av + 8)/16;
   old_av = new_av;
   return new_av;
}

If you rewrite this for a float variable, you should leave out the '+8'.

If you want to reverse the order,

Serial.print(arr[(index + i) % n]);

can be replaced with:

Serial.print(arr[(index + n - i - 1) % n]);

As for naming conventions, push_back / pop_front or put / get would indeed be better, as mentioned by @alto777.

Thanks for the extra input @jfjlaros but I'm afraid coding that way is above my skill level. The use of templates is totally new to me for example.

I thought this was a sub-question in my project but I see there is some confusion about what I am trying to do. Like an X-Y problem.

Therefore I decided to post my project "tutorial style" to give insight into the why:
HDD Motor Scrollwheel

The functions are templated only to pass the array and its size in one go. If the array and its size are passed separately, non-templated functions can be used.

Yes, I noticed it a few minutes ago. Very nice and creative idea.

Yes because those are stack (FILO) terms. A circular buffer is usually a FIFO.

Ok,

So I've been looking into circular buffers, an interesting and useful concept I can use in the future.
For my current project I think I'll stick with the code I have:
HDD Motor Scrollwheel - working but room for improvement]

@jfjlaros Sorry man, thanks for the help but your code is just too advanced for me at this point

I did however write code based on the circular buffer concept. It works and fits my need. I'll give implementing it into my project a thought. See the code below.

Thanks for the input to all for now, I've learned enough.

The only thing I have not figured out is what '% n' does or means. I could not google "%n" so what is that method / notation called? Thanks.

#include <Streaming.h>

#define BUFLEN 5                // Length of buffer

int buf[BUFLEN];                // Define buffer
int writeIndex = 0;             // Index at which a new value should be written to
int readFrom;                   // Start index of reading 
long multiplier;                // Used in get() to caculate result, long so BUFLEN can be 5 or more
long result;                    // Returned from get()
int val;                        // New value to be stored 


void setup()
{
  Serial.begin(115200);
}


void loop()
{
  val = random(1, 7);
  
  put(val);
  
  Serial << "New value has been put in: " << val << endl;

  Serial << "The values in sequence of input: "  << endl;
  
  Serial << get() << endl;
  
  delay(1500);
}


void put(int item)
{
  writeIndex++;                                         // Increment write index
  
  if(writeIndex == BUFLEN) writeIndex = 0;              // Set back to zero if 'out of index' is reached
  
  buf[writeIndex] = item;                               // Save/store value
}


long get()                                              // Long because return can be > 65000
{
  readFrom = writeIndex;                                // Start reading from the index of the last written value 
  
  multiplier = 1;                                       // Set multiplier for the first read value
  
  result = 0;                                           // Reset result to zero

  for(int i = 0; i < BUFLEN; i++)                       // Read entries of whole buffer
  {    
    result += buf[readFrom] * multiplier;               // Buffer 641, result: 1*1 + 10*4 + 100*6 = 641
    
    multiplier = multiplier * 10;
    
    readFrom--;                                         // Go backward through the buffer
    
    if(readFrom == -1) readFrom = BUFLEN - 1;           // If index < 0 got to end
  }
    
  return result;
}

% is the modulo operator and essentially does the job of your if statement and increment in this context.

a7

This calculates the remainder of a division by n.

E.g.,

Serial.println(5 % 7);  // Prints "5".
Serial.println(6 % 7);  // Prints "6".
Serial.println(7 % 7);  // Prints "0".
Serial.println(8 % 7);  // Prints "1".

Thanks,

I used the modulo before, but I think I'l have to try and find my 'N'
I'm guessing right now that BUFLEN is involved.

I'll go and chew on it ok :wink:

Things like '%' you don't "figure out". You look them up.

It's safer to enforce the bound:

if(writeIndex >= BUFLEN) writeIndex = 0;

This approach seems clumsy to some programmers but it's actually orders of magnitude faster than modulo arithmetic.

Another nice thing can happen if the size is a power of two - just use & (and) to mask off the bits that aren't part of the number:

 writeIndex++: writeIndex &= 0x1f;    // N = 32

works both ways with unsigned integers.

Big win: buffer size is the same as the number of values in the integers used for indexing, like 256.

If you scrounging for us microseconds.

a7

1 Like

Thing is, with a FILO you are always doing an increment with any read or write, so it's a perfect opportunity to test and enforce the boundaries because you can combine the wrap around with the increment. You have to do the increment anyway, whether you use modulo wrap or "test and correct" wrap. If you know what I mean...