max length of an array?

Forum 2005-2010 (read only) Software Bugs & Suggestions
1

I have an int array holding 160 single digit values that are either 1, 2 or 0. In fact there are three of these arrays -all the same. I've been having what I thought were overflow problems so I made a smaller program and sent the values to the serial port with a loop.
I only write to a few of the locations in the array using firstArray[position] = 1; where position is always between 0 and 159.
So there is no fear of overflowing the array.
When I get the serial print out, the second array comes back as expected: 160 values that are perfect. But the first and third arrays come back intact only to about 148 positions and then i get an additional strand of nonsense numbers, including multiple negative signs that extend way out past the 160 values.
I have tried putting delays in the serial print loop thinking that the port was drinking from a fire-hose, but no dice.
I don't think it's a serial.print issue or a serial port buffer size because this is exactly the problem my larger program was symptomatic of.

Any thoughts?

2

If you have three arrays of that size, you are consuming 23160 = 960 bytes of the 1024 byte of RAM available just for these arrays. One very simple fix would be to change the type from "int" to "byte".

Mikal

3

Yes we were coming to the same conclusion ourselves.
Since my values don't get anywhere near 255, I was thinking 4 bits was what was being stored, even though I was allocating enough for 16.
We're going to change the int arrays to byte arrays but what I'd really like is a "bit" datatype, so that each entry is a bit - either off or on. I would need more arrays to provide enough description but at a 16th the size of the current ones I think I could afford it.

Just not used to cramming stuff into small packages!

Thanks

4

I have an int array holding 160 single digit values that are either 1, 2 or 0.

Since the values you're storing have a very limited range, you could easily store 4 values per byte, i.e., using two bits for each value. Using this strategy, you could store 160 values using only 40 bytes of RAM.

The example code below (untested, but it compiles) implements this idea using two functions, getVal() and putVal() for accessing the sub-elements.

static const unsigned bitsPerElem = 2;  // this should be a factor of 8 for best results
static const unsigned elemPerByte = 8 / bitsPerElem;
static const unsigned elemMask = (1 << bitsPerElem) - 1;

uint8_t getVal(uint8_t *array, unsigned arraySize, unsigned idx)
{
  uint8_t val = 0;
  uint8_t subIdx = idx % elemPerByte;
  idx /= elemPerByte;
  if (idx < arraySize)
    val = (array[idx] >> (subIdx * bitsPerElem)) & elemMask;
  return(val);
}

void putVal(uint8_t *array, unsigned arraySize, unsigned idx, uint8_t val)
{
  uint8_t subIdx = idx % elemPerByte;
  idx /= elemPerByte;
  if (idx < arraySize)
  {
    // remove the old value, install the new value
    array[idx] &= ~(elemMask << (subIdx * bitsPerElem));
    array[idx] |= (val * elemMask) << (subIdx * bitsPerElem);
  }
}

This code fragment shows how it might be used:

uint8_t myArray[20];
int val, idx;
idx = 27;
val = getVal(myArray, sizeof(myArray), idx);
putVal(myArray, sizeof(myArray), idx, 2);

Of course, this idea can be much more elegantly implemented by defining a new class. I'll leave that as an exercise for the interested reader.

5

Here is kind of a kludgy bit array, with the bitVal macro you can address individual bits in a byte array. Weather it saves any space or not depends on how often you use the macro (might be better as a function) and how many bits you need to reference:

byte bitvals[] = {B10000010,B00000001};

#define bitVal(x,y) ((y[x/8] & 1<<(7-(x%8))) != 0)

void setup(){
  Serial.begin(9600);
}
void loop(){
   Serial.println("bitvaltest");
   Serial.println(bitVal(0,bitvals));
   Serial.println(bitVal(2,bitvals));
   Serial.println(bitVal(6,bitvals));
   Serial.println(bitVal(14,bitvals));
   Serial.println(bitVal(15,bitvals));
   while(true);
}
6

So I've gone to a byte array (just to keep things simple and to keep the clock cycles down from extra operations - and I really need 5 distinct values per position). And it seems to solve the problem.
But here's a gotcha:
I had spaces in my 160 element array so I could visually see the breaks at every 16 elements.
So: byte myArray[] ={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0 .....
With these spaces in place (which should get compiled out) I still get the over flow errors.
And I have tried spaces before the numeral and after just for fun but it gives a similar but different error.
Take the spaces out and all is well.

Thanks for all the help guys.

7

I'm dubious about the effect that the spaces could possibly have on the runtime code. Could you perhaps post the code so that we can take a closer look?

Meanwhile, if "array" is a global, you don't need to initialize it to 0; that's the default.

Mikal

8

Meanwhile, if "array" is a global, you don't need to initialize it to 0; that's the default.

sorry, that's wrong. C has no automatic initialization; a variable will contain whatever garbage happens to be in memory when it is allocated.

There may be some design feature of the ATmega that results in zeroed memory on powerup or something, but it would be poor programming practice to count on it in general.

-j

9

C has no automatic initialization; a variable will contain whatever garbage happens to be in memory when it is allocated.

While your assertion is true for dynamically allocated variables (those defined in a function but without the static attribute), it is most definitely not true for statically allocated variables which, according to the C and C++ standards, are guaranteed to be initialized to zero in the absence of an explicit initialization.

Moreover, the documentatioin for the gcc compiler used for the Arduino specifically recommends against specifying zero initializers for statically allocated variables because it unnecessarily consumes code space to do so.

10

No, it IS true, KG. C and C++ guarantee that all objects with static duration are initialized to 0 (unless of course explicitly initialized to something else). As you point out, variables with "auto" class (i.e. stack) are not provided with this guarantee, because to do so would entail a performance penalty. But since initializing statics/globals doesn't incur any such performance hit, the compiler provides the initialization for you. (It was explained to me, once upon a time, that Kernighan and Ritchie tried to balance performance with predictibility, but that when these collided, performance always won out.)

Alas, I'm traveling now, and cannot eloquently quote from my ANSI C or Stroustrup C++ books to substantiate this, but I'm pretty sure you'll find it if you look. :slight_smile:

Mikal

[Edit: Sorry, Don. Didn't see your post... thanks!]

11

Ah well, wrong again. I learned C before there was an ANSI standard. |:

That's a bit unfortunate, though. If initialization behaves one way for automatic variables and a different way for static, it would seem to make it more error prone for the programmer.

I was trained to assign a value to a variable before I used it, period. That style has served me well for too many years to try to change it...

-j

12

Though I will say I am a big fan of default values as they have their benefits (less clutter and potential typos). I'm quite used to a compiler warning (in java) when a local variable goes uninitialized. You learn to initialize those pretty quick. There is a gcc setting to warn you of this as well:

-Wuninitialized

13

I learned C before there was an ANSI standard. |:

That's a bit unfortunate, though. If initialization behaves one way for automatic variables and a different way for static, it would seem to make it more error prone for the programmer.

I was trained to assign a value to a variable before I used it, period. That style has served me well for too many years to try to change it...

FWIW, the rules for initialization of static variables did not change when C went through ANSI. C documentation has stated from the start that static variables are always guaranteed to be initialized to zero 'conceptually at compile time' (see chap 4.9 of the pre ANSI version of K&R)

But both the pre ansi and ansi versions of K&R show examples with static variables being explicitly set to zero in a program. So if that's a style K&R use in their code then you are in good company :wink:

14

But both the pre ansi and ansi versions of K&R show examples with static variables being explicitly set to zero in a program.

In many years of C/C++ programming I always explicitly initialized statically allocated variables as a matter of course even if the value was zero. I've had to break myself of that habit when writing C/C++ code for the AVRs due to the undesirable effect of wasting limited code space.

For more information on this issue, see the avr-gcc FAQ entry on the topic.

Note: On reviewing the FAQ entry above, I see that the avr-gcc compiler has been modified (since I first began using it) so that there is now no code space penalty for explicit zero initialization.

15

-Wuninitialized

I generally use -Wall which subsumes -Wuninitialized. Some may find -Wall a bit annoying and/or confusing, though.

16

...due to the undesirable effect of wasting limited code space.

Don, I'm having a hard time understanding why the addition of the zero initializer would waste code space. Shouldn't the compiler be smart enough to optimize [global] "int x = 0;" into the equivalent "int x;"?

In Arduino-land at least, the answer seems to be yes. I did a little experiment with three different versions of the same program, each using an array x.

// sketch size 2836
int x[20];

// also size 2836: no change
int x[20] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};

// sketch bigger! size 2876
int x[20] = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20};

Mikal

17

Note: On reviewing the FAQ entry above, I see that the avr-gcc compiler has been modified (since I first began using it) so that there is now no code space penalty for explicit zero initialization.

Don, its good to hear that the compiler is handling that now, thanks for posting

Mikal, the compiler used to put the code to initialize a variable to zero in flash. It now just moves the declaration into the .bss segment.