array of variable names

Hi all, Is there a way to store an array of variable names, then iterate through that array assigning values to the actual names of the variables?

To expand, I have a struct containing all the config variables for a project I’m working on (home automation device), and I write that struct to eeprom, sd card, allow the user to restore from sd, have a html gui editor for the values etc.
My struct already contains the names and sizes and types of each element, but I’m ignoring that information and recreating it each time. While it works, its error prone to debug, there’s lots of replication in each subsystem which deals with the struct contents and its ugly, and I’m doing this to learn and improve.
So ideally I’d like a way to iterate over the entire struct, then assign individual elements of it to a value. Its the strncpy(Configuration.myProjectName,writeval,16); that I am struggling with how to do the assignment of without typing Configuration.myProjectName as the destination which changes for each element.

void readConfig(String readString,Client &readclient) {
  myFile = SD.open("/config.txt");
  // if the file opened okay, read config from it:
    if (myFile) {
      while (myFile.available()) {
            fileLine="";
        
            while (!(fileLine.indexOf('\n')  > 0)){   //while we're not on a newline boundary
              fileLine += (char(myFile.read()));   
              }
      if ( fileLine.startsWith(F("myProjectName"))) {fileLine.remove(0,14); readclient.print (F("Setting myProjectName to ")); readclient.println (fileLine); 
          char writeval[16]; fileLine.toCharArray(writeval, fileLine.length() + 1); strncpy(Configuration.myProjectName,writeval,16); }
      if ( fileLine.startsWith(F("myVersionChar"))) {fileLine.remove(0,14); readclient.print (F("Setting myVersionChar to ")); readclient.println (fileLine); 
          char writeval[16]; fileLine.toCharArray(writeval, fileLine.length() + 1); strncpy(Configuration.myVersionchar,writeval,4); }
             }
//Ad nauseum until all the struct elements I want to restore back are covered
    myFile.close();
  }
}

And my struct looks like this

typedef struct {
    // The variables of your settings
  char myProjectName[17]; 
  char myVersionchar[5];
  char myLocName[17];
  int numRelays;
  byte tftDisplay; // If we're running a tft display and want output on it
  byte debug;
  byte remotelog; //This just makes a request to a remote webserver so it gets logged in the httpd access log of that machine
  char extServer[17]; //This is used also for the images for the web gui
  char extServerURL[17];  ;// ? is the url termination string char, padded to allow long urls on edit
  int extServerPort; 
  char unitName[16][15]; 
  byte remotejson;
char extJsonServer [17];
int extJsonServerPort; 
char JsonPassword[17];
byte dht11Present; ;// DHTxx sensors present (0 or 1)
char dhtsensorName[16][15]; ;// Sensor name to return, padded to 14 chars
int dataPinSensor[16][1]; ;// Array of pins the sensors are located at
byte BinSensorPresent; // Config for if binary input sensors present (PIR, rain sensor etc
char BinSensorName[16][15]; // Sensor name to return padded to 14 chars
byte dataPinBinSensor[16][1]; // Array of pins the sensors are located at
byte numSensors;
byte numBinSensors;
byte mac[6];
byte ipAddress[6];
int httpListenPort;
byte myTimeServer[6];
int touchpressure;
byte editModeUnlocked;
char editModePassword[17];
// This is for mere detection if they are your settings
  char version_of_program[5]; // it is the last variable of the struct
  // so when settings are saved, they will only be validated if
  // they are stored completely.
} Configuration_t;

You could maybe use the stringyfy preprocessor operator to get you roughly where you want to be, but at runtime, there's no such thing as a variable name.

Are you trying to all user configuration of your device at run time?

How do you see the user selecting the options? It sounds like you are considering some form of web UI or screen/menu system...

explaining how you plan to allow user configuration may help with some constructive advice.

BulldogLowell, yes, the devices (arduino mega's) "boot" and check their eeprom for config during init, and while running allow the user to edit the config items via a web UI config editor (unlocked by entering the correct editModePassword into the web UI), then the user gets to choose to burn the struct back to eeprom or just use it until the device is restarted eg if you have turned the debug level up but next power cycle you might not want to commit that level to long term storage. The same struct can be saved off to sd card, and that file downloaded via the web UI to store off device.

Ive done the webUI already, but its a awful lot of print's and even with some defines to remove common elements its still overweight. I did define a second array of arrays so I could iterate or use a case statement to switch but it consumed so much dram I went back to individual prints from flash ram space.
Maybe I could store a array of pointers to strings stored in flash ram space or something.
I'm just currently working on the bit that reads the config back in from sd card and applies it to memory and wondered if I might find a better way.

Each arduino device is running a w5100 ethernet card and a mcufriend tft display with a sd card in its slot, and controls a configurable amount of relays (up to 16), and sensors (16 dht and 16 binary types), which it exports off to a home-assistant home automation console, but also maintains a local web UI to permit switching direct from the local subnet (for those situations when the home-assistant computer is down), plus it takes local inputs from hardwired switches, it sends all of these methods by json off to the home assistant controller to keep it all in sync.

The idea is once they're flashed, I don't need to trudge out to a outbuilding to change or configure them, and I have config backups which change all the variances between locations. And their config can be adjusted via the webUI by other people without having to go near the code.
At the moment I have to go out with a usb cable and a laptop to reflash with newer code because I've not looked at if/how that's possible with a new bootloader yet...

Its 1700 lines of code so far not including the configuration and inline image declarations, but I'm trying to neaten up and simplify it before publishing it under some kind of open license because frankly, there's a lot of horrible things going on at the moment without bounds checking or much sanitization of input and I'm a bit embarrassed at the code quality of it.

MrFluffy:
Its 1700 lines of code so far not including the configuration and inline image declarations, but I'm trying to neaten up and simplify it before publishing it...

so with a small program like that, I may take a different approach...

Store your basic user configuration into EEPROM on your initial flash.

From the UI, all of your options (from the custom config) will simply store the new value into EEPROM when user sets the value from say an options list.

The various settings are stored in FLASH, saving a new configuration setting merely updates the EEPROM.

Software restart to latch the new settings once user is done (i.e. selects "Confirm new Settings" in your UI).

easy.

AWOL:
You could maybe use the stringyfy preprocessor operator to get you roughly where you want to be, but at runtime, there's no such thing as a variable name.

Its runtime location I'm trying to get to.

What I'm after achieving (I think, to write it out) is a area of memory where a load of memory locations are stored which point to other memory locations where the struct contents are stored.
A structure table of struct locations but not sure that's the correct term.
So I can say iterate over that master list and if what memory locations its pointing to matches a condition, then do the conditional rather than doing it as individual lines in my if's. But to format that into something the compiler will understand that's what I want.
How that deals with the struct I have now being non uniform in size, would mean that the amount of bytes each bit would have to be stored somehow so the processor would know what offset or memory location the next bit of config will be found out. Or maybe the compiler would deal with that when it was writing the locations of the master struct entries in.

Does that make any sense or is my thinking broken somewhere?

BulldogLowell:
so with a small program like that, I may take a different approach…

Store your basic user configuration into EEPROM on your initial flash.

From the UI, all of your options (from the custom config) will simply store the new value into EEPROM when user sets the value from say an options list.

The various settings are stored in FLASH, saving a new configuration setting merely updates the EEPROM.

Software restart to latch the new settings once user is done (i.e. selects “Confirm new Settings” in your UI).

easy.

Hmm I hadn’t considered doing it that way, thanks. That way all the “variables” which only change during config could be stored in flash and save the entire dram space consumed at runtime by the struct.
Then they could be copied into dram from the eeprom as the authoritive source for editing and saving back to eeprom and done as individual items rather than writing the whole struct once the editing has finished.

But, that doesn’t simplify my original problem of having a location of locations to edit and extract the information from, I think.
I probably need to go away and think this through a bit more in terms of memory locations to get it straight in my own head what I’m trying to achieve…

copied into dram SRAM for editing and saving back to flashEEPROM

MrFluffy:
Does that make any sense or is my thinking broken somewhere?

No, it seems you are explaining your proposed programmatic solution rather than what you are simply trying to do.

My home automation thingy can behave like a Monkey, a Rabbit a Chicken or a Chameleon. I want to allow user at run-time to select from the list of four options...

If user selects Monkey, he can choose from Resus, Lesula, or Mandril... kind-of-thing.

AWOL, thanks for picking that up, I edited the post before reading your correction after thinking it through better or I wouldn’t have edited it.

BulldogLowell: I’m not after a nested config, its just my terrible explanation. What I would like to get to is my home automation thingy is default called Monkey, and its number of legs is set to 2, and those legs are called left and right.
I want to allow my user at runtime to be able to change the species and redefine the number of legs up to a maximum of 16, and give each of the new legs new names to identify them in gui elements, and that they might have two states, up and down.
I also want the user to be able to look at the produced config files in a text editor and say “Oh look the species is set to tripod_alien and it has 3 legs called middle,left,right when it was saved” without needing machine interpretation.

In my real world example there’s a perl program that reads these backed up configs (actually currently it interrogates the web UI via a series of CURL requests but that requires all devices to be online during exec to be valid) and produces a home-assistant configuration file from them so the json element names match up both on the device and the home-automation console and if I edit the amount of legs and add a 4th leg called outer_left and run the perl script, it appears in the home-assistant console immediately on exec and allows people to lower and raise the leg.

I think I can see my worst mistake, those variable names are blown away at compile time and replaced by memory locations.
I want to work with the names post compile so I want to store a look up table of ascii strings of the variable names , and what memory address inside the struct the variable name at compile time is stored at. I could use this human friendly representation where it was useful to convey meaning and look it up in the look up table when I want to do something to the contents of that address, and for the 2d arrays I would know from the struct definition how many bytes the next element would be offset from the base address, so being able to deconstruct eg unitName[4] from base location of unitName+amount of bytes each row consumes (16).

So I can then scan for the ascii string in configuration files, and change that memory location to be the value in the eeprom so next boot it gets copied into flash ram and uses the updated representation.

I want to work with the names post compile so I want to store a look up table of ascii strings of the variable names

Some useful search terms in reply #1

MrFluffy:
Hi all, Is there a way to store an array of variable names, then iterate through that array assigning values to the actual names of the variables?

No, but you can have an array of pointers to variables. You can then iterate through that and assign values to the variables.

gfvalvo:
No, but you can have an array of pointers to variables. You can then iterate through that and assign values to the variables.

right!

#include <EEPROM.h>

#define MAX_NUM_ANIMALS 5

enum Species{
  MONKEY,
  LLAMA,
  CHICKEN,
  TIGER
};

struct Animal{
  Species species;
  int numLegs;
};

Animal* animal[MAX_NUM_ANIMALS]; // create array of pointers to animal objects, unfortunately we have to define its size here, otherwise we would have used a vector.
int numAnimals = 0;  // initialize with zero elements

void setup() 
{
  EEPROM.get(0,numAnimals); // query for number of animal objects, you saved that at EEPROM location zero
  for(int i = 0; i < numAnimals && i < MAX_NUM_ANIMALS; i++) // now create a new 
  {
    animal[i] = new Animal();
    EEPROM.get(1+i*sizeof(Animal), animal[i]); // added sizeof over struct iteration
  }
}

void loop() 
{
  // put your main code here, to run repeatedly:

}

Or, just assign all your storage at compile time and avoid dynamic allocation (new).

#include <EEPROM.h>

#define MAX_NUM_ANIMALS 5

enum Species{
  MONKEY,
  LLAMA,
  CHICKEN,
  TIGER
};

typedef struct {
  Species species;
  int numLegs;
} Animal;

Animal animal[MAX_NUM_ANIMALS]; // create array of Animal structs, 'animal' is a pointer this arrary
int numAnimals = 0;  // initialize with zero elements

void setup() 
{
  EEPROM.get(0,numAnimals); // query for number of animal objects, you saved that at EEPROM location zero
  for(int i = 0; i < numAnimals && i < MAX_NUM_ANIMALS; i++) // now create a new 
  {
    EEPROM.get(1+i*sizeof(Animal), animal+i); // added sizeof over struct iteration
  }
}

void loop() 
{
  // put your main code here, to run repeatedly:

}

Ok so as not to be a drive by, I looked at the above examples, but my issue is species (or location in my real world use) isnt a predetermined list, or defined in advance. Anything could go in there so I cant declare a fixed enum and list all the types.
AWoL, I looked at tidying things up with compile time macro's, I've already done that but it makes it a nightmare to read back if you make a syntactic mistake. I'm simple, I need all the help I can get in making things clear.

Going back to the original idea, while it might not be at all the best way, Ive done this

const char varName1[] PROGMEM = "myProjectName";
const char varName2[] PROGMEM = "myVersionchar";
const char varName3[] PROGMEM = "myLocName";

---* more lots of declarations abbrieviated for clarity *---

PGM_P const varName_table[] PROGMEM= { varName1,varName2,varName3 };

Then the incoming config string is read and compared against this list of ascii representations based in progmem.

String myvariableTypeName=strcpy_P(progmemBuffer, (char*)pgm_read_word(&(varName_table[i])));
          if (fileLine.startsWith(myvariableTypeName)){ 
//Do stuff to the value
}

Which is ok, until I need to strcpy it back into somewhere to reapply it. So if I have a matching array of memory locations to map those to, I could use that instead. It'd be nicer as a multi dimensional array with both types in it, but I was in keep it simple stupid mode running on coffee late at night.

char  *varNameLoc_table[]= {&Configuration.myProjectName[0],&Configuration.myVersionchar[0],&Configuration.myLocName[0],};

which lets me map back the values to the original entered value to test its working/make sure the user wants to apply them instead.

Serial. println (&*(varNameLoc_table[i]));

Cunning flaw in this butchery is that I've used a array not a struct, and some of my memory locations are pointing to int or byte or other types, so I can't declare their memory locations in a regular array as the pointers have to be the same type.
I'm going to beaver away at storing them in a struct instead so I can mix the types and try to work the pointer logic out to get them back out.

Its horrible, but I'm getting my head around c pointers better now, I think.