Importing a large lookup table

int table [100] [1] ;

As AWOL was suggesting, a table dimension of 1 isn't much use.

table [0] [0] = x0 ;
table [0] [1] = y0 ;
table [1] [0] = x1;
table [1] [1] = y1;

No. Assign the values like this:

int table [100] [2] = {
  { 1, 2 },
  { 3, 4 },

// and so on

  };

And then keep it in flash memory with the PROGMEM directive so it doesn't use any RAM.

eg.

int PROGMEM table [100] [2] = {
  { 1, 2 },
  { 3, 4 },
  { 5, 6 },

// and so on

  };

Thanks AWOL and Nick,
I am still baffled by the fact that in the declaration "1" means 1 total and in the assignment "1" means the second item: 0 - 1.
I keep on making this mistake and the IDE keeps on reminding me about that (I'm glad the IDE didn't get tired of that yet).

Thanks also for the PROGMEM tip and for the link! I can do it so long as the program does not need to modify it, correct?
Though it may not be necessary as an int array of 100 x 2 entries should occupy 2 x 100 x 2 = 400 bytes).
As the second numbers are not bigger than a byte I could create two monodimensional arrays, one int and one byte for a total of 100 * (2 + 1) = 300 bytes (plus some overhead, I assume).

Finally, can I put the code in a function at the end of the sketch and call it from setup(), maybe by stating somehow it's a global array or, being it a variable assignment must it go before setup() (that would make the sketch neater) ?

Thanks

Thot:
I am still baffled by the fact that in the declaration "1" means 1 total and in the assignment "1" means the second item: 0 - 1.

You declare the number you want. eg.

int foo [2];

You get an array of two. However they are zero-relative. So you index into 0, 1.

Thanks also for the PROGMEM tip and for the link! I can do it so long as the program does not need to modify it, correct?

Yes.

Finally, can I put the code in a function at the end of the sketch and call it from setup(), maybe by stating somehow it's a global array or, being it a variable assignment must it go before setup() (that would make the sketch neater) ?

The assignments would take instructions. It seems much simpler to pre-assign the array (ie. declare the contents, don't assign them). Then it would need to be globally defined (or at least, static inside a function).

As the second numbers are not bigger than a byte I could create two monodimensional arrays, one int and one byte for a total of 100 * (2 + 1) = 300 bytes (plus some overhead, I assume).

You could do that, but if you keep them in program memory you have 32 Kb to play with, so you aren't running out yet.

In search for the way to import an array of numeric data from a separate file into my sketch, I found a post in a c++ forum that showcased this code:

static const char* data[] = {  // <--- new line 
#include "mydata.hh" 
};   // <--- don't forget semi-colon

I put in "mydata.hh" the following text:
"tizio", "caio", "sempronio"

and created the following sketch:

static const char* data[] = {  // <--- new line 
#include "E:\Downloads\Devices\Arduino\arduino-1.0.1\data\mydata.hh" 
};   // <--- don't forget semi-colon

void setup() {
  Serial.begin(9600);
}

void loop () {
  Serial.print (data[0]);
  Serial.println();
  
}

It compiles flawlessly but the serial monitor shows nothing (should show "tizio")!

Then I tried this:

static const int data[] = {  // <--- new line changed char* to int
#include "E:\Downloads\Devices\Arduino\arduino-1.0.1\data\numbers.hh" 
};   // <--- don't forget semi-colon

void setup() {
  Serial.begin(9600);
}

void loop () {
  Serial.print (data[0]);
  Serial.println();
  
}

with numbers.hh containing:
1,2,3,4

It also compiles flawlessly but the serial monitor shows nothing (in other attempt it showed numbers totally unrelated to the content of the file)!

I feel I am very close on learning how to import numeric data from an external file but I am unable to walk the final mile.

BTW, Arduino reference says that using pointers is very complicate and I probably don't need them anyways :frowning:

Any help is appreciated

TIA

How about posting the file? Or at least the first few lines of it.

It should work, are you sure the baud speed is the same in the serial monitor window?

How big is this file, anyway? How many numbers are in it?

It compiles flawlessly but the serial monitor shows nothing (should show "tizio")!

On what version of the IDE? Prior to 1.0.1, the failure of the compiler to find an include file was a fatal error. With 1.0.1, the failure to find an include file doesn't even generate a warning.

The failure to find the include file would explain the behavior you are seeing.

Eventually, my goal is to import a bidimensional array of numbers, but right now, for testing purposes I am experimenting with the following:
The entirety of "mydata.hh" contains the following text:

"tizio", "caio", "sempronio"

The entirety of "numbers.hh" contains:

1,2,3,4

"mydata.hh" on disk is 30 bytes long.
"numbers.hh"on disk is 9 bytes long.

guix:
It should work, are you sure the baud speed is the same in the serial monitor window?

Yes I made sure they are the same.

PaulS:
With 1.0.1, the failure to find an include file doesn't even generate a warning.
The failure to find the include file would explain the behavior you are seeing.

I am using 1.0.1
If I don't put the whole path, the compilers gives me an error: "No such file or directory".

Thanks for your help.

Ok, this is maddening!

This morning, after posting the reply above I re-run the sketches without changing anything and they work!

One thing that I did not mention is that yesterday, after changing something here and there and re-verifying I was getting a bunch of random errors that did not make much sense, such as that I already declared variables that were not even in the sketch anymore.....

Next I will actually try to import my large bidimensional array with the same method and I may reach for help again, but thank you for taking the time to answer.

One thing: does Arduino work better in daytime?
Any problem with full moon? :slight_smile:

I was getting a bunch of random errors

I've never heard of a compiler producing random errors.

I've never heard of a compiler producing random errors.

Incomprehensible, infuriating, annoying, yes. Random, no.

PaulS:

I've never heard of a compiler producing random errors.

Incomprehensible, infuriating, annoying, yes. Random, no.

Perhaps random was not the right term, incomprehensible is better.

I was getting something like (I don't remember it verbatim and the error did not show up again, this morning):
variable char[] already defined
when I never defined a variable called char[].
In a previous sketch I had defined
char* data[]

Maybe there is some reason for it {the pointer was still in the computer/arduino memory ??)
But if there is it is definitely not obvious :0

The entirety of "numbers.hh" contains:

That wouldn't be the right way to go.

You need two files:

numbers.h:

//header file for numbers

extern const unsigned char mydata[];

numbers.c/ino/pde

#include "numbers.h"

const unsigned char mydata[]={
  1, 2, 3, 4,
  ...
};

In your user application, you simply include numbers.h in order to reference mydata[].

Ok, I think I did it.
I found a way to keep a large array of data in a separate text file and importing it in my sketch as an #include.
The advantages of doing that are:

  • you can modify and update your data file without touching the sketch.
  • you don't have to put an unsightly long declaration in your code.
  • you don't even have to re-verify your code after modifying the data but you have to re-upload it

I thought this was a fairly established thing to do but I could find hardly any documentation.
I am posting my solution here hoping it may be helpful to some other newbie.

/* This examples illustrates how to import numeric data from a separate Text file

   By Luca Brigatti - 2012 
   witht he help of the folks in the Arduino programming forum.

   Note: Leaving the first dimension empty: data[][2]
   and using: sizeof(data) 
   to calculate the size of the array in bytes eliminates the need to know beforehand 
   how many values are there in the file.
   
   The operation: maxIndex = sizeof(data) / 4 - 1
   return the highest index number of the first dimension:  data[maxIndex][2]
   The denominator (4 in this case) is the product of the number of bytes per data type (2 bytes per integer) 
times the size of the second dimension ([2] in this case)
   
   The text file being read: "numbers2.hh"
   contains something like that: 
   ________________________
   // Example of bidimensional numeric data array
   // The actual numbers can be any int (if the data array is int)
  {1,1},
  {-2,4},
  {3,-9},
  {4,}, // Second element here will be 0, Like:  {4,0},
  {-5,-25},
......
_________________________________

Note: {,4},   i.e leaving the first place empty is NOT allowed.  
*/

// Main importing instructions. data[][2] can be a variable
static const int data[] [2] = {  
// Put the name of the file with the complete Path
#include "E:\Downloads\Devices\Arduino\arduino-1.0.1\data\numbers2.hh" 
};

int maxIndex ;
int arraySize ;

void setup() {
  
  Serial.begin(9600); // Initiate serial
  
  arraySize=sizeof(data);  // Gets the size of the array in bytes
  maxIndex = arraySize/4 - 1; // Size /2 (bytes in an int) /2 (for a bidimensional array) - 1 (as index starts at 0)   
  
  // Print dimensions of the array
  Serial.print("Array Size (bytes): ");
  Serial.println (arraySize);
  Serial.print ("# of rows: ");
  Serial.println(maxIndex+1);
  
  delay(2000);
}

void loop () {
  
  // Print all the data in the array, over and over again
  for (byte i=0; i<=maxIndex; i++) {
    Serial.print (data[i][0]);
    Serial.print (" , ");
    Serial.println (data[i][1]);
  }
  delay(500);
    Serial.println();
  
}
  • you don't even have to re-verify your code after modifying the data but you have to re-upload it

Yeah, right. Perhaps you want to enable verbose output when you compile, but not when you upload. Then, see what happens when you click Upload.

"I am posting my solution here hoping it may be helpful to some other newbie."

That approach works if the data is referenced from one source file.

If you want to reference the data from multiple files, you have to split it into .c and .h files.

If you want to reference the data from multiple files, you have to split it into .c and .h files.

Of course, it would be a lot easier if you used .cpp and .h files.

dhenry:
That approach works if the data is referenced from one source file.

Yes, that was exactly my goal:
ONE file, separate from the sketch itself, containing all my data.

Besides, nothing would prevent me to use another #include for a different set of data:

static const int data[] [2] = {  
#include "E:\DataPath\numbers2.hh" 
};

static const byte other_data[] [3] = {  
#include "E:\DataPath\other_numbers.hh" 
};