Best way to store many variables (multi-dimensional array?)

Projects Programming
1

I put some code together on an Arduino MEGA to read a Modbus interface, which provides some 100 values, which are then published via MQTT
The code runs nicely for a few months now.

However, the interface is polled every 10 seconds, which results in 600 values per minute, or 850,000 per day. A bit ridiculous, in particular noting, that some of these values do not change for days, while others update every second.

My idea is to store all values in an array, its register address, and a divisor. E.g., like so:
addr, old, divisor
8000, 12345, 10
8001, 23456, -10
8018, 9876, 1

The values are both int16 and uint16, stipulating I need two arrays?!
Or store all values as unsigned and convert to signed based on boolean flag?!
E.g.
addr, old, divisor, signed
8000, 12345, 10, 1
8001, 23456, -10, 0
8018, 9876, 1, 0

The register addresses are non-consecutive, and may not be queried at each loop.

I have never worked with multi-dimensional arrays, and wondered about their performance in C, or if there is a better way of doing this,

So why store the values? So that I can compare these, and only output a value, when the value has changed.

What is also not clear how to find the register address, which is not equal to the array index, and is also not sorted. The latter is beyond control as I could read a different amount of addresses per loop.

Should I create the array with the register addresses, given that I also need a divisor value per register as well as a boolean flag for signed int16?

Any hints appreciated.

An example of reading 17 registers:

void read_inverter ()
{
    static const uint16_t start_address = 0x1f4f;   // 8015
    static const uint8_t number_of_registers = 17;  // 8031
    static uint8_t result = 1;

    result = modbus.readHoldingRegisters(start_address, number_of_registers);

    if (result == modbus.ku8MBSuccess)
    {
        if (1 == OUTPUT_VALUES_VIA_MQTT)
        {
            // AC_Inverter_Power
            g_register_value_signed = (int16_t)modbus.getResponseBuffer(0x00);
            g_register_address = start_address + 0;
            data_publish_int(g_register_address, g_register_value_signed, 10);

            // AC_Inverter_Current
            g_register_value_signed = (int16_t)modbus.getResponseBuffer(0x01);
            g_register_address = start_address + 1;
            data_publish_int(g_register_address, g_register_value_signed, -10);

            // Inverter_Reactive
            g_register_value_signed = (int16_t)modbus.getResponseBuffer(0x02);
            g_register_address = start_address + 2;
            data_publish_int(g_register_address, g_register_value_signed, 10);

            // Output_Status
            g_register_value = (uint16_t)modbus.getResponseBuffer(0x03);
            g_register_address = start_address + 3;
            data_publish_uint(g_register_address, g_register_value, 1);

            uint8_t i;

            for (i = 4; i <= 6; i++)
            {   // temp
                g_register_value = (uint16_t)modbus.getResponseBuffer(i);
                g_register_address = start_address + i;
                data_publish_uint(g_register_address, g_register_value, -10);
            }

            g_register_value = (uint16_t)modbus.getResponseBuffer(0x07);
            g_register_address = start_address + 7;
            data_publish_uint(g_register_address, g_register_value, 1);

            for (i = 8; i <= 11; i++)
            {   // limits
                g_register_value_signed = (int16_t)modbus.getResponseBuffer(i);
                g_register_address = start_address + i;
                data_publish_int(g_register_address, g_register_value_signed, 10);
            }

            for (i = 12; i <= 14; i++)
            {   // status
                g_register_value = (uint16_t)modbus.getResponseBuffer(i);
                g_register_address = start_address + i;
                data_publish_uint(g_register_address, g_register_value, 1);
            }

            g_register_value_signed = (int16_t)modbus.getResponseBuffer(0x0f);
            g_register_address = start_address + 15;
            data_publish_int(g_register_address, g_register_value_signed, 10);

            g_register_value = (uint16_t)modbus.getResponseBuffer(0x10);
            g_register_address = start_address + 16;
            data_publish_uint(g_register_address, g_register_value, 1);
        }
    }
    else
    {
        decode_error(result, start_address);
    }

    return;

}   /* read_inverter () */

Yes, I am aware of the code smell, but have not found a better way yet.
I imagine the multi-dimensional array may allow for a function to emerge that simplifies the code.

[edit1] added board (MEGA)

2

You should maybe consider an array of structs rather than 2 arrays

1 Like
3

My preference would be using a struct. I am not sure if you are asking how to reduce your memory footprint or how to keep track of each data field etc. but a struct will sort it all out for you. What's more, you can use bit-fields in a struct so that you only need as many bits as the range you need. Order your fields within the struct from largest to smallest in size to optimize padding. As for signed variables, if you need a signed variable, use a signed variable. Keep in mind, bit-fields work best with unsigned variables. And, you can have an array of structs, so organize your "fields" variables in the struct for one record, and then just have an array of that struct for as many records as you will need.

1 Like
4

not sure what processor you're using, but it's unlikely that it has enough storage for this amount of data

since you mentioned only storing changes, maybe just save a delta timestamp

what do you mean by register?

i must be missing something, why pre-populate? is the divisor per value or only for specific values? and why do you need a flag for a signed value, the most significant bit is the sign

i know my furnace monitor isn't on the same scale as what you're doing, but it only records a limited # of events (furnace on/off times). my struct is a timestamp, pin and state. i have to upload data often enough to capture all data.

5

Values are not stored but published via MQTT...

Modbus terminology... a register contains a value.

create an array with 100 rows...

see next

Thanks for the explanation.
struct: never used one, and am figuring out to use this. :slight_smile:
This works for a concept...

#include <stdio.h>
#include <stdint.h>

#define NUM_MODBUS_ARR_ITEMS  6

struct modbus_data
{
    // int8 -128 to 127
    // uint8   0 to 255

    // int16 -32,768 to 32,767
    // uint16      0 to 65,535

    uint16_t register_address;
    uint8_t value_signed;
    uint8_t factor;             // 0 = none, 1 = 0.1, 2 = 10
    uint16_t value;
};

struct modbus_data arr_modbus_data[NUM_MODBUS_ARR_ITEMS] =
{
    {8000,1,1,12340},
    {8001,1,1,15678},
    {8002,1,2,234},
    {8003,1,1,3},
    {8004,1,2,42},
    {8005,1,1,52345}
};

int main ()
{
    for (uint8_t i = 0; i < NUM_MODBUS_ARR_ITEMS; i++)
    {
        printf("%d %d %d %d\n",
            arr_modbus_data[i].register_address,
            arr_modbus_data[i].value_signed,
            arr_modbus_data[i].factor,
            arr_modbus_data[i].value);
    }
}

Result:

8000 1 1 12340
8001 1 1 15678
8002 1 2 234
8003 1 1 3
8004 1 2 42
8005 1 1 52345
6

why do you need to store anything if it can be "published" right away?

7

I'm glad you're all sorted out now.

arrays all same data type, which makes it easy to access them with a pointer or index, reference and dereference etc. If you need this kind of access AND your working with the same data type, use a multi-dimensional array.

structs varied data types, best for accessing several different data types as related.

8

Don't forget a plug for arrays of structs.

a7

9

see below:

1 Like
10

My O.P. says, I only want to publish when the value has changed compared to the previous one. Hence, I need to store it.

11

Then you don't need an array because you are only saving one set of variables

12

Not that simple (I j think) I need to store 106 values to compare.

And I store the int type and factor per address as well; meaning code can be simplified.

13

because there are that many registers?

so there's a some value for something(s). a new value is compared to the old and "published" if different, as well as updating the current "last" value

14

consider something like this
(stubs & all compiles)

struct Modbus {
    int16_t getResponseBuffer (int n) { return 0; }
} modbus;

void data_publish_int (uint16_t adr, int16_t val, int16_t fact) { }

// -----------------------------------------------------------------------------
const uint16_t BaseAddr = 0x1f4f;
const uint16_t Nregs    = 17;

int16_t values  [Nregs] = {};

const int16_t Factors [Nregs] = {
      0, -10,  10, 1,
    -10, -10, -10,      // 4-6
      1,
     10,  10,  10, 10,  // 8-11
      1,  1,  1,        // 12-14
     10,  1
};

void read_inverter ()
{
    for (unsigned n = 0; n < Nregs; n++)  {
        int16_t val = (int16_t)modbus.getResponseBuffer (n);

        if (values [n] != val)  {
            values [n] = val;
            data_publish_int (BaseAddr + n, val, Factors [n]);
        }
    }
}

// -----------------------------------------------------------------------------
void
loop (void)
{
}

void
setup (void)
{
    Serial.begin (9600);
}
1 Like
15

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.