Hi, I need to be able to reference a lookup table based on temp and relative humidity. At a given relative humidity and temperature 2 values are given (abs. g/m3 and VPD kPa ) I'm just wondering how best to store this information. I was thinking nested arrays. Any pointers appreciated
Hi, I need to be able to reference a lookup table based on temp and relative humidity.
How many rows (temperature) and columns (humidity)? A 4D table will fill the Arduino's memory pretty quickly.
Temp is 5 (10,15,20,25,30)
RH is 11 (30,40,50,60,70,75,80,85,90,95,100)
Each cross ref having two fixed values.
Full table bottom of this page http://www.hortnet.co.nz/publications/science/n/neder/humid02.htm
May have to pad RH due to shifting scale adding 35,45,55,65?
May have to pad RH due to shifting scale adding 35,45,55,65?
The extrapolation math would be a lot simpler if all the RH steps were the same size.
The table index values would also be easier to calculate.
You don't actually store the temp and RH values in the lookup table. They are used to determine the row and column that contains the actual data of interest.
So, 2 tables containing 75 values would hold 150 values. The values are floats, so 4 bytes per value. The tables would consume 600 bytes. The 328-based Arduinos have 2K of SRAM, or you can store the tables only in PROGMEM, and save that SRAM, if it is in short supply.
This has the potential to keep growing (no pun intended if you figured out what I'm working on) so wouldn't hurt to store in PROGMEM now. I can't see there being a need to re reference oftern with a 5degC change in temp or 5% RH being required to change the outcome.
I need to look into PROGMEM first. So I'm thinking 5 arrays (temp) with 15 elements (RH) each holding a nested array of the two values. Does that make sense?
I'm not sure how to constrain the sensor values to the scale yet either.
So I'm thinking 5 arrays (temp) with 15 elements (RH) each holding a nested array of the two values. Does that make sense?
No. While you might have an array of Temp values and an array of RH values, they are used only to compute indices into the other two arrays, containing the other two sets of values.
Where does the 5th array come in?
And, no, none of them are 2D (or nested, whatever that means) arrays.
Just create a single dimension array and then index into it to retrieve your two values.
You have 5x temperature columns, 11x RH rows & 2x values per entry so to to read the two values from row 5 (80% RH), column 3 (20C) just do something like
abs = table [((RH - 1) * 10) + ((temperature - 1) * 2) + 0]
VPD = table [((RH - 1) * 10) + ((temperature - 1) * 2) + 1]
const float table[]{
9.42, 0.0, 12.86, 0.0, 17.33, 0.0, 23.09, 0.0, 30.43, 0.0,
8.94, 0.06, 12.21, 0.09, 16.47, 0.12, 21.94, 0.16, 28.91, 0.21,
8.47, 0.12, 11.57, 0.17, 15.60, 0.23, 20.79, 0.32, 27.39, 0.42,
8.00, 0.18, 10.93, 0.26, 14.73, 0.35, 19.63, 0.48, 25.87, 0.64,
7.53, 0.25, 10.28, 0.34, 13.87, 0.47, 18.84, 0.63, 24.34, 0.85,
7.06, 0.31, 9.64, 0.43, 13.00, 0.59, 17.32, 0.79, 22.82, 1.06,
6.59, 0.37, 9.00, 0.51, 12.13, 0.70, 16.17, 0.95, 21.30, 1.27,
5.65, 0.49, 7.71, 0.68, 10.40, 0.94, 13.86, 1.27, 18.26, 1.70,
4.71, 0.61, 6.43, 0.85, 8.67, 1.17, 11.55, 1.59, 15.22, 2.12,
3.77, 0.74, 5.14, 1.02, 6.93, 1.41, 9.24, 1.90, 12.17, 2.55,
2.82, 0.86, 3.86, 1.20, 5.20, 1.64, 6.93, 2.22, 9.13, 2.97
};
PaulS:
So I'm thinking 5 arrays (temp) with 15 elements (RH) each holding a nested array of the two values. Does that make sense?
No. While you might have an array of Temp values and an array of RH values, they are used only to compute indices into the other two arrays, containing the other two sets of values.
Where does the 5th array come in?
And, no, none of them are 2D (or nested, whatever that means) arrays.
5th array is after the 4th array of values for each temp. So I would have an array for 10degC containing elements for each RH that is actually another array of the two values of interest.
nested means what you clearly thought it meant. An array within an array element, ie a list within a list. Or if you prefer '2D'
For what it's worth, here is my code so far. Indecently I changed my mind on how to factor the code opting for a 3D array, temp,humidity & values of interest:
//RH/TEMP VPD&Abs Humidity conversion.
float vpd_table[5][15][2] = {
//30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
/*10 degC*/{ {0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0} },
/*15 degC*/{ {0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0} },
/*20 degC*/{ {0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0} },
/*25 degC*/{ {0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0} },
/*30 degC*/{ {0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0},{0.0,0.0} }};
// END VPD table
Riva:
Just create a single dimension array and then index into it to retrieve your two values.
Ah, I see what you mean. Thank you! Is there a memory or performance benefit to either solution?
How do you display memory usage with the Arduino platform?
dannix:
Is there a memory or performance benefit to either solution?How do you display memory usage with the Arduino platform?
I don't think there will be much of a difference performance wise and the memory usage should be about the same though you did talk of putting it in program memory. See here http://www.arduino.cc/playground/Main/PROGMEM for good example.
Have a look here Arduino Playground - HomePage for info about memory usage.
Riva:
I don't think there will be much of a difference performance wise and the memory usage should be about the same though you did talk of putting it in program memory. See here http://www.arduino.cc/playground/Main/PROGMEM for good example.Have a look here Arduino Playground - HomePage for info about memory usage.
FreeMemory() reports 960 with both mine and your table commented out, 960 with my table only and 960 with both tables present. What makes a difference is const, which I added to my declaration too once I'd seen your use of it. What exactly is const doing besides making it illegal to assign a value to it in run time? When adding PROGMEM keyword there is no change either. Is const moving it to different ram also?
To clarify, SRAM decreases without const in the declarations. SRAM remains the same whether the tables are in PROGMEM or not and defined or not.
What exactly is const doing besides making it illegal to assign a value to it in run time? When adding PROGMEM keyword there is no change either. Is const moving it to different ram also?
Maybe some smarter person than me can answer this better/fully/more accurately.
const (short for constant) is just telling the compiler this value is unchanging. Maybe it automatically stores constants in PROGMEM or maybe float is not a type that can be stored in PROGMEM. I have only ever used PROGMEM for char & byte arrays and it did increase program size and I assumed increased program size means arrays were also in program memory.
reading the PROGMEM reference float isn't supported, or at least the required info isn't given for float. Not a big problem I can multiply by 100 or whatever to loose the float requirement. However if defined in progmem or as a const or not a const the sketch size reported is the same, which is I believe stored in progmem so suggests in all cases the tables are not in progmem for what ever reason.
I need to look into memory management more
reading the PROGMEM reference float isn't supported
..but reading unsigned longs is, and a simple cast would sort the rest out.
AWOL:
a simple cast would sort the rest out.
can you elaborate? I'd noticed long and as I said I could convert them to alternative format by multiplying by 100 or whatever to remove the decimal place.
AWOL:
reading the PROGMEM reference float isn't supported
..but reading unsigned longs is, and a simple cast would sort the rest out.
Anyone else wondering what "a simple cast" is look here: http://www.arduino.cc/en/Reference/Cast It actually won't do what's is required. As it simply drops the floating amount, 0.45 and 0.75 would both be 0 after the cast or am i missing something here?
dannix:
reading the PROGMEM reference float isn't supported, or at least the required info isn't given for float.
I've had success storing lots of floats in progmem as follows (apologies for the giant table, it's copy-pasted from my existing code base):
#include <avr/pgmspace.h>
typedef float PROGMEM prog_float_t; // Need to define this type before use
// Stick a 10-row, 37-column array of float values into program memory
PROGMEM prog_float_t Equilarg[10][37] = {
{174.95,18.51,216.55,157.82,231.31,194.25,291.37,28.5,222.75,57,41.93,249.62,171.62,53.64,349.93,19.3,226.99,176.92,180,0,0,0,3.08,29.59,26.6,178.91,156.28,212.76,9.99,236.18,194.25,165.75,31.01,165.75,152.32,280.07,200.13},
{275.37,17.7,214.57,348.14,159.57,270.19,225.29,180.38,90.58,0.77,16.09,121.98,248.92,333.61,349.19,354.82,100.72,177.65,180,0,0,0,2.35,287.37,178.91,73.02,51.75,287.89,162.69,286.28,270.19,89.81,312.35,89.81,254.1,280.81,201.62},
{0.02,14.86,208.97,182.74,72.78,10.35,195.53,20.71,31.06,41.42,27.53,44.7,353.16,221.23,349.43,10.33,27.51,177.39,180,0,0,0,2.61,198.06,19.83,2.65,345.46,25.21,5.85,37.88,10.35,349.65,204.04,349.65,342.83,280.57,201.15},
{83.38,11.3,202.4,3.66,322.99,110.42,165.63,220.84,331.26,81.68,38.87,327.32,98.58,105.09,349.67,27.03,315.49,177.14,180,0,0,0,2.86,108.65,220.64,292.19,280.36,121.72,209.54,149.29,110.42,249.58,93.25,249.58,71.55,280.33,200.67},
{166.42,7.56,195.61,182.09,211.44,210.46,135.69,60.92,271.39,121.85,50.19,249.92,204.31,348,349.9,44.04,243.77,176.88,180,0,0,0,3.12,19.21,61.44,221.71,215.56,218.02,53.36,260.65,210.46,149.54,341.84,149.54,160.27,280.1,200.19},
{264.33,5.26,191.35,1.96,102.4,286.18,69.27,212.36,138.54,64.72,24.12,122.06,283.91,260.62,349.16,21.85,119.79,177.61,180,0,0,0,2.39,276.77,213.53,115.59,113.32,291.44,207.11,310.3,286.18,73.82,258.36,73.82,262.06,280.84,201.69},
{349.75,2.87,186.44,202.17,42.75,26.41,39.61,52.82,79.23,105.64,35.63,44.85,27.45,150.48,349.4,36.66,45.88,177.35,180,0,0,0,2.65,187.53,54.51,45.29,46.33,29.28,49.95,62.04,26.41,333.59,151.52,333.59,350.78,280.6,201.21},
{76.92,1.58,183.41,39.83,314.62,126.83,10.24,253.65,20.48,147.31,47.32,327.82,129.41,45.42,349.63,49.91,330.4,177.1,180,0,0,0,2.9,98.47,255.68,335.19,337.77,128.41,252.07,174.15,126.83,233.17,48,233.17,79.5,280.37,200.73},
{165.67,1.37,182.52,211.35,206.58,227.47,341.21,94.94,322.41,189.88,59.24,251.02,230.02,304.87,349.87,61.8,253.57,176.84,180,0,0,0,3.16,9.64,97.08,265.3,267.86,228.84,93.57,286.71,227.47,132.53,307.42,132.53,168.23,280.13,200.25},
{269.75,3.07,185.62,17.26,90.25,303.97,275.96,247.95,191.92,135.9,33.96,123.94,304.24,235.22,349.13,34.23,124.21,177.57,180,0,0,0,2.43,267.98,249.95,159.97,160.24,307.04,244.88,337.93,303.97,56.03,235.49,56.03,270.01,280.87,201.75}
};
void setup() {
Serial.begin(115200);
}
void loop(void) {
float retrieval;
for (int row = 0; row < 10; row++) {
Serial.print("Row: ");
Serial.println(row + 1);
for (int col = 0; col < 37; col++) {
retrieval = pgm_read_float_near(&Equilarg[row][col]);
Serial.println(retrieval,4);
delay(1000);
}
}
}
That's just one table from my library, I have two 10x37, and three more 37-value float tables, which should add up to ~3404 bytes in memory. They all fit on a 328 with no problem. They had no chance of fitting in the regular SRAM when I tried that originally.
Someone with more experience than me can explain the pgm_read_float_near and "&" syntax used to retrieve the values for use in the main loop.
Thank you very much!
How odd that information isn't available in the PROGMEM reference. Assuming it works, which you say it does then all that is different is defining the type of data and it's retrieval function.
typedef float PROGMEM prog_float_t; // Need to define this type before use
The & just means pass a reference to the data not the actual data. http://arduino.cc/it/Reference/Pointer It needs de-referencing where it's used which will be in the pgm_read_float_near() using (the * syntax) which I assume is provided by including avrprogmem or does your code have a function called pgm_read_float_near in it?
dannix:
The & just means pass a reference to the data not the actual data. http://arduino.cc/it/Reference/Pointer It needs de-referencing where it's used which will be in the pgm_read_float_near() using (the * syntax) which I assume is provided by including avrprogmem or does your code have a function called pgm_read_float_near in it?
Nope, no extra functions in my code. The code I posted should work as a self-contained example.