Lookup table: PROGMEM vs. EEPROM vs. generate lookup dynamically

I have two lookup tables in my program. Here is the procedure that generates the lookup table values:

void populateLookupArrays() 
{
  int y;

  for (y = 0; y < (maxPWMOut + 1); y++)          // Populate the linear map
    linearMap[y] = y;                            // Well, that was easy.

  // Populate the "accelerated" map. This map has a sharp upslope for a short percentage of the pedal's travel,
  // then a more gradual slope once you get closer to the desired welding amperage.
  // Both slopes are linear, there is just a "knee" in the line. The inflection point is defined by rise1 and run1.

  const int rise1 = ((maxPWMOut) * 75) / 100;                   // 75% of the welder's output
  const int run1 = ((maxPWMOut) * 25) / 100;                    // ... will be achieved in the first 25% of pedal travel
  const float slope1 = (float)rise1 / (float)run1;
  const int rise2 = maxPWMOut - rise1;                          // parameters for the second, "gradual" half of the travel--derived from rise and run
  const int run2 = maxPWMOut - run1;
  const float slope2 = (float)rise2 / (float)run2;
  const float yIntercept2 = maxPWMOut - (slope2 * maxPWMOut);

#ifdef DEBUG_POPULATELOOKUPARRAYS  
  Serial << "populateLookupArrays():";
  Serial << "\trise1 = "<< rise1;
  Serial << "\trun1 = " << run1;
  Serial << "\tslope1 = " << slope1;
  Serial << "\trise2 = " << rise2;
  Serial << "\trun2 = " << run2;
  Serial << "\tslope2 = " << slope2;
  Serial << "\tyInt2 = " << yIntercept2;
  Serial.println();
  Serial.println("Populating acceleratedMap...");
  Serial.println("First for loop...");
#endif

  for (y = 0; y < run1; y++) 
  {
    acceleratedMap[y] = round((float)y * slope1);
#ifdef DEBUG_POPULATELOOKUPARRAYS
    Serial << y << ", " << acceleratedMap[y] << endl;
#endif
  }

When I was developing the code, I had the size of the lookup tables set to 101, but I later increased them to 256, so that there was one entry in the table for each possible PWM output value (they are looking up PWM outputs). This caused me to run out of SRAM. So now I need to figure out what to do next.

I can think of four possibilities:

  1. Use PROGMEM.
  2. Write the tables to EEPROM and get them from there.
  3. Calculate the lookup value in real-time.
  4. Print out the table values, then manually enter them into a constant array. Will this allow the compiler to optimize them such that they don't take up SRAM?

I suspect that, of the three, option (3) is probably the right one, although it is frankly also the most annoying for me to write. Option (2) has the issue that if the code is loaded onto a chip that hasn't had the EEPROM staged, or with an out-of-date lookup table in the EEPROM, the code won't work right, so I think that is probably out of the question. I don't know much about PROGMEM...

joshuabardwell:

  const int rise1 = ((maxPWMOut) * 75) / 100;                   // 75% of the welder's output

const int run1 = ((maxPWMOut) * 25) / 100;                    // ... will be achieved in the first 25% of pedal travel

PS: I have been warned that putting values into the comments like that is bad practice. This code is from before that warning :wink:

... now that I look at it, I also have no idea why maxPWMOut is in parentheses.

Your linearMap appears to be a waste of space. Anywhere you have linearMap[i] replace it with i

Pete

el_supremo:
Your linearMap appears to be a waste of space. Anywhere you have linearMap[i] replace it with i

You're right, in the specific case of LinearMap, but to do what you suggest, I think I would have to write an exception into the part of the code that is doing the lookup.

byte getPedalOutputPWM() 
{
  int pedalPotAnalog = aMux.AnalogRead(pedalPotPin);                            // read actual pedal position
  byte pedalPotPWM = map(pedalPotAnalog, 0, numAnalogSteps, 0, maxOutputPWM);   // now map that to a PWM value between 0-maxOutput
  byte outputPWM = pedalTravelLookupArray[pedalPotPWM];                         // now look up the appropriate output PWM value as given by the current pedalTravelMode lookup table
  return outputPWM;
}

I'd rather have the details of the lookup be compartmentalized from the function that does the looking-up--no?

... I guess I could handle it by writing a function that returned the appropriate lookup value, depending on travelMode. That function could simply return "x" if the travel mode is linear, or acceleratedMap[x] if it's accelerated. But at that point, I'm halfway towards writing the whole thing procedurally, and I should probably just ditch lookup tables entirely. Frankly, the lookup tables won't scale anyway. Even if two of them would fit in EEPROM or PROGMEM, a larger number might not. And the tradeoff in terms of speed for a calculation vs. a lookup is probably negligible.

Okay, so:

byte pwmOutputMapping_Accelerated(byte i) {
  const static int rise1 = (maxPWMOut * 75) / 100;                   // 75% of the welder's output
  const static int run1 = (maxPWMOut * 25) / 100;                    // ... will be achieved in the first 25% of pedal travel
  const static float slope1 = (float)rise1 / (float)run1;
  const static int rise2 = maxPWMOut - rise1;                          // parameters for the second, "gradual" half of the travel--derived from rise and run
  const static int run2 = maxPWMOut - run1;
  const static float slope2 = (float)rise2 / (float)run2;
  const static float yIntercept2 = maxPWMOut - (slope2 * maxPWMOut);
  
  if (i < run1)
    return round((float)y * slope1)
  else
    return round(((float)y * slope2) + yIntercept2);
}

Is this an appropriate use of the static modifier? Or is it totally redundant due to the const? My thinking is that the variables are never going to change after compile-time, so I am trying to avoid having to re-calculate all of them each time the function is called.

... on a related note, I should really get those floats out of there. TODO.

If you're just going for a dual slope approach, you don't need lookup tables. Just store the coordinates of your "knee" in a pair of variables and use map() to interpolate the values in between. in fact, you already do that in rise1 and run1. I would choose better names for them though. In my example I'll use knee_x and knee_y. I know I'll be mixing styles, but I like underscores.

const int MaxPWMOut = 255;
const int MaxADCOut = 1023;

byte getPedalOutputPWM() 
{
  int pedalPotAnalog = aMux.AnalogRead(pedalPotPin);                            // read actual pedal position

  byte outputPWM;

  if ( pedalPotAnalog < knee_x )
    outputPWM = map( pedalPotAnalog, 0, knee_x, 0, knee_y );  // Interpolation below the knee
  else
    outputPWM = map( pedalPotAnalog, knee_x, MaxADCOut, knee_y, MaxPWMOut ); // Iterpolation above the knee
  
  return outputPWM;
}

NOTE: This assumes that you change knee_x to use MaxADCOut (1023) instead of MaxPWMOut. It'll be easier that way. No need to use a lookup table or anything complicated for this.

If you have more than one knee, or want to scale with a different function that isn't linear, a lookup table can be made. But that's another topic.

joshuabardwell:
I can think of four possibilities:

  1. Use PROGMEM.
  2. Write the tables to EEPROM and get them from there.
  3. Calculate the lookup value in real-time.
  4. Print out the table values, then manually enter them into a constant array. Will this allow the compiler to optimize them such that they don't take up SRAM?

Using PROGMEM for the array is what you want, and you'll have to generate
the values to do this:

#include <avr/pgmspace.h>

const int acceleratedMap [256] PROGMEM = 
{ 0, 1, 10, 34324, ....
};

[ guessing at the array type ]

If your mapping isn't constant you'll have to consider EEPROM, but there's only
1k, whereas PROGMEM is 32k (assuming a 328)

Jiggy-Ninja:
If you're just going for a dual slope approach, you don't need lookup tables. Just store the coordinates of your "knee" in a pair of variables and use map() to interpolate the values in between. in fact, you already do that in rise1 and run1. I would choose better names for them though. In my example I'll use knee_x and knee_y. I know I'll be mixing styles, but I like underscores.

...

If you have more than one knee, or want to scale with a different function that isn't linear, a lookup table can be made. But that's another topic.

Thanks for that. I didn't even think of using map() for this, although I see that it's perfect. Here I was doing all that math... Your second comment reminds me that the original reason why I started using lookup tables is that I wanted to do a logarithmic curve, but once I started actually fiddling with logs, I realized that simple linear graph with a knee would work better.

Here's the code as it stands now. Spot checks at min, max, and knee show correct values, with sensible results between. Thanks for your suggestion. This makes the code much simpler and more compact.

const byte maxPWMOut = 255;
const int numAnalogSteps = 1024;

// accelerated mapping -- linear with a "knee" -- sharp upslope then long tail at the top
byte pwmOutputMapping_Accelerated(int  analog) {
  const int knee_x = (long)numAnalogSteps * 25 / 100; // this percent of pedal travel ...
  const int knee_y = (long)maxPWMOut * 75 / 100;      // ... will result in this percent of the welder's output

  if (analog > numAnalogSteps - 1)
    analog = numAnalogSteps;

  if (analog < knee_x)
    return map(analog, 0, knee_x, 0, knee_y); // Interpolation below the knee
  else
    return map(analog, knee_x, numAnalogSteps, knee_y, maxPWMOut + 1); // Interpolation above the knee
}

// linear mapping
byte pwmOutputMapping_Linear(int analog) {
  return map(analog, 0, numAnalogSteps, 0, maxPWMOut + 1);
}

void setup() {
  Serial.begin(9600);
  delay(1000);
}

void loop() {
  for (int x = 0; x < numAnalogSteps; x++)
  {
    Serial.print(x);
    Serial.print("\tlinear = ");
    Serial.print(pwmOutputMapping_Linear(x));
    Serial.print("\taccelerated = ");
    Serial.print(pwmOutputMapping_Accelerated(x));
    Serial.println();
  }
  delay(500);
}

And here is the getPedalOutputPWM() function:

// Returns the output PWM value that is correct for the pedal's current position,
// allowing for maxOutput limitation and for different pedal response curves
byte getPedalOutputPWM() 
{
  int pedalPotAnalog = aMux.AnalogRead(pedalPotPin);                                                // read actual pedal position
  int scaledAnalog = ((((long)maxOutputPWM * 1000) / maxPWMOut) * (long)numAnalogSteps) / 1000;     // now scale that down to 0...maxOutput
  
  // now map that to the appropriate value for our travelMode
  switch (pedalTravelMode) {
  case travelModeLinear:
    byte outputPWM = pwmOutputMapping_Linear(scaledAnalog);
    break;
  case travelModeAccelerated:
    byte outputPWM =  pwmOutputMapping_Accelerated(scaledAnalog);
    break;
  default:
    byte outputPWM = pwmOutputMapping_Linear(scaledAnalog);
    break;
  }

#ifdef DEBUG_PEDAL
  Serial << "getPedalOutputPWM():";
  Serial << "\tpedalPotAnalog = " << pedalPotAnalog;
  Serial << "\tmaxOutputPWM = " << maxOutputPWM;
  Serial << "\tpedalPotPWM = " << pedalPotPWM;
  Serial << "\toutputPWM = " << outputPWM;
  Serial.println();
#endif

  return outputPWM;
}

The reason for scaledAnalog is that the welder's output is not always intended to go from min to full-range. Rather, we want to be able to map the pedal's travel from min to some percentage of the welder's full range, such that, for example, 100% of pedal travel = 50% of welder's range. Therefore, we must scale down analog before passing it to the "mapping" functions. I had been doing this by scaling full-range analog -> full-range pwm, then scaling full-range pwm -> limited-range pwm. But I think that keeping the numbers in the "analog" domain as long as possible makes sense, because you keep more significant figures longer (10-bit vs. 8-bit).

Where is pedalPotAnalog used in the getPedalOutputPWM() function? I can't see it. I don't understand what scaledAnalog is for.

Jiggy-Ninja:
Where is pedalPotAnalog used in the getPedalOutputPWM() function? I can't see it. I don't understand what scaledAnalog is for.

Oh sheesh. This is what I get when I write code and then post it before actually debugging. The problem is that the hardware this device plugs into is currently "in transition" so I can't easily debug the code. Rather than post a "corrected" version that might also have a mistake, I'll just say that the intent is:

  1. Read the pedal position.
  2. Take the value of another pot (maxOutputPWM). This value scales the welder's output down from full-range.
  3. Scale the pedal position down based on the value of maxOutputPWM--i.e. maxOutputPWM = 128 (50%), pedalPotAnalog = 512 (50%) => pedalPotAnalogScaled = 256 (25%).
  4. Look up pedalPotAnalogScaled in a lookup table to derive a PWM output value. In the case of linear travel mode, this is just map(analog, 0, 1024, 0, 256). In the case of accelerated travel mode, it's the "knee" method.

... okay ... I think this should be correct:

int scaledAnalog = ((((long)maxOutputPWM * 1000) / maxPWMOut) * (long)pedalPotAnalog) / 1000;

And BTW, I acknowledge how confusing it is to have two variables/consts named maxPWMOut and maxOutputPWM. I like to put units after my variable names, when appropriate, which is how maxOutputPWM came about: the welder's maximum output level, represented as a PWM output value. Vs, for example, maxOutputAnalog, which would be the analogRead() value returned by the pot that is used to enter the maxOutput parameter. maxPWMOut was originally named pwmResolution, but I realized that technically the resolution should be 8 or 10--the number of bits of resolution--so that was incorrect. Finally I realized that the value I was trying to represent was the maximum PWM output value, so that's what I named it. Anyway, it makes sense to me, but every time I think about somebody else looking at it, I realize how confusing it probably is and think I should change it.

Okay, this should be a little better even still, I think.

  const int pedalPotAnalog = aMux.AnalogRead(pedalPotPin);                                 // read actual pedal position
  const int precisionMultiplier = 1000;                                                    // add significant figures to non-float math
  const long maxOutputRatio = ((long)maxOutputPWM * precisionMultiplier) / maxPWMOut;      // calculate ratio of welder max output to max possible (full-range) output
  const int scaledAnalog = (maxOutputRatio * (long)pedalPotAnalog) / precisionMultiplier;  // scale pedalPotAnalog down by maxOutputRatio and back out the precisionMultiplier

joshuabardwell:
Oh sheesh. This is what I get when I write code and then post it before actually debugging. The problem is that the hardware this device plugs into is currently "in transition" so I can't easily debug the code. Rather than post a "corrected" version that might also have a mistake, I'll just say that the intent is:

  1. Read the pedal position.
  2. Take the value of another pot (maxOutputPWM). This value scales the welder's output down from full-range.
  3. Scale the pedal position down based on the value of maxOutputPWM--i.e. maxOutputPWM = 128 (50%), pedalPotAnalog = 512 (50%) => pedalPotAnalogScaled = 256 (25%).
  4. Look up pedalPotAnalogScaled in a lookup table to derive a PWM output value. In the case of linear travel mode, this is just map(analog, 0, 1024, 0, 256). In the case of accelerated travel mode, it's the "knee" method.

This is a good description. Gives me an idea of what you're doing.

  1. Call it welderOutputLimit, or something similar. Much more descriptive than what you have.

  2. Rather than scaling the pedal position, I think you'd be better off using welderOutputLimit to scale the y coordinates (the output values), and leave your x values the same.

For linear:

outputPWM = map( pedalPotAnalog, MIN_PEDAL, MAX_PEDAL, 0, welderOutputLimit );

I use MIN_PEDAL and MAX_PEDAL constants because your pedal might not (probably does not) sweep the full ADC range. It might read 200 when unpressed, and 900 fully pressed, for example. Those are values you'll have to find out by using it.

For knee, you might scale the knee's y location like this:

const float kneeYPercentage = 0.75; // This should be up at the top with your other constants.
const float kneeXPercentage - 0.25;
const int kneeX = (MAX_PEDAL - MIN_PEDAL) * kneeXPercentage + MIN_PEDAL;

int scaledKneeY = welderOutputLimit * kneeYPercentage

if ( pedalPotAnalog < knee_x )
    outputPWM = map( pedalPotAnalog, MIN_PEDAL, kneeX , 0, scaledKneeY );  // Interpolation below the knee
  else
    outputPWM = map( pedalPotAnalog, kneeX , MAX_PEDAL, scaledKneeY , welderOutputLimit ); // Iterpolation above the knee

This code is made under the assumption that you want the output limit to vertically scale the whole output transfer function. Like in the Excel graph that I attached a picture of. In that graph, the output is set to 75% of full scale, shown by the red line. Is that the kind of thing you want. The way you posted it, it looks like the scaled transfer function will have a different shape. It'll be fine for linear, but for the accelerated one (the one with the knee, I assume), the knee will no longer be at 25% travel. Try modelling your functions on a spreadsheet graph and see if they work fine.

You know, I think I had that same "doesn't scale" bug when I originally wrote the function, and I thought I fixed it. Maybe I re-introduced it somewhere along the line.