Go Down

Topic: Importing a large lookup table (Read 2470 times) previous topic - next topic

dhenry

Quote
ONE file, separate from the sketch itself, containing all my data.


That's not the point.

Your approach wouldn't work if multiple files make wants to reference the data in "numbers.hh".


Quote

Your approach wouldn't work if multiple files make wants to reference the data in "numbers.hh".


Sorry, that I didn't understand.
Once I have my data in an array, I can reference that in the sketch, I won't need to reference "numbers.hh" anymore.

There are three kind of people in the world: Those who can count, and those who can't

Nick Gammon

Quote
Importing a large lookup table



"mydata.hh" on disk is 30 bytes long.
"numbers.hh"on disk is 9 bytes long.


Your idea of "large" and mine must differ.
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

guix

It was probably a small test file just to demonstrate the problem... :)


Your idea of "large" and mine must differ.



It was probably a small test file just to demonstrate the problem... :)


Indeed :)
I made up a couple of quick text files, one with strings, the other with integers, to see if I could make the darn thing work!

And when I finally made it to work (again, I did not do anything different from last night, today it just worked) it also works with the "real" thing.

It turns out that I was able to reduce the actual table, by organizing the data in a different way to a mere 400 bytes or so, well below the RAM restrictions.

Small size still, probably by your (and my) standards. :P
There are three kind of people in the world: Those who can count, and those who can't

Nick Gammon

You might be able to put the table into PROGMEM to minimize your RAM usage.

Also, the compiler tries to not compile things it doesn't think have changed. I'm not sure, but it's possible that if you change the .h file it won't realize to recompile the .cpp file. In other words, if you change the numbers but not the sketch.

I don't like #includes in the middle of a declaration, personally. Why not just put the whole array into a separate file (including the declaration, and the trailing brace)?
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics


I don't like #includes in the middle of a declaration, personally. Why not just put the whole array into a separate file (including the declaration, and the trailing brace)?


I thought of doing something like that but I don't know how to do it  :smiley-roll-sweat:
Even thought of creating a "Library" but when I consulted the reference on how to do that I thought it was a bit of an overkill for my purpose.

Also, my way (well, the one I learned form a c++ forum) does not require two files (.h and .cpp) but just one: .hh  (and by the way, I have no clue of the meaning of the .hh extension ....) which would seem to me more practical.

But again, I don't have much experience and I would not be surprised if there were a better way. :)

Come to think of it, is the .h and .cpp business the same as creating a library?
There are three kind of people in the world: Those who can count, and those who can't

Nick Gammon

Just put all the stuff in a separate .h file, eg. from my bootloader project:

Code: [Select]

byte PROGMEM atmega328_optiboot [] = {

0x11, 0x24, 0x84, 0xB7, 0x14, 0xBE, 0x81, 0xFF, 0xF0, 0xD0, 0x85, 0xE0, 0x80, 0x93, 0x81, 0x00,

... blah blah ...

0xFF, 0x27, 0x09, 0x94, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x04, 0x04,

};  // end of atmega328_optiboot
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

dhenry

Quote
I don't know how to do it


I posted an example for you earlier.

You should never declare a variable in a .h file - that's so un-c that it will get you fired in most places. Declaring a variable in .h files creates localized global variables at each inclusion of those .h files.

Instead, declare the variable in the .c file and if such variables are to be referenced somewhere else, "extern" such variables.

#39
Nov 21, 2012, 10:34 pm Last Edit: Nov 21, 2012, 10:40 pm by Thot Reason: 1
@Nick

So, if I understand correctly:

In a separate file called mydata.h  I put:

Code: [Select]
const int data [] [2] = {
    {1,2},
    {3,4},
    {5,6},
// and so on...

} ;


(assuming it's small enough to fit in RAM)

And then in the main sketch I put:

Code: [Select]
#include "mydata.h" ;


Is it that simple?
That would have the additional advantage that I can put several arrays in the same file (if I need them) and just use an #include in the main sketch.

What about the mentioned extra .cpp file?


There are three kind of people in the world: Those who can count, and those who can't

#40
Nov 21, 2012, 10:38 pm Last Edit: Nov 21, 2012, 10:41 pm by Thot Reason: 1

I posted an example for you earlier.

You should never declare a variable in a .h file - that's so un-c that it will get you fired in most places. Declaring a variable in .h files creates localized global variables at each inclusion of those .h files.

Instead, declare the variable in the .c file and if such variables are to be referenced somewhere else, "extern" such variables.



Yes, thank you. I saw your earlier post but I did not fully understand that, sorry :)

So, the example I just asked Nick about is not correct because I AM declaring the variable in an .h file?

Thanks

P.s. I'm glad I am not a programmer, finding a job these days is a pain ....
There are three kind of people in the world: Those who can count, and those who can't

Nick Gammon

dhenry has got a point, however his suggested change has its own issues.

Let's make an example.

Sketch:

Code: [Select]

void setup ()
  {
  Serial.begin (115200);
  Serial.println (data [0] [0]); 
  }  // end of setup
 
void loop ()  {  }



foo.cpp:


Code: [Select]

const int data [] [2] = {
     {1,2},
     {3,4},
     {5,6},
// and so on...
} ;


Error:

Code: [Select]

sketch_nov22a.cpp: In function 'void setup()':
sketch_nov22a:5: error: 'data' was not declared in this scope


So the variable "data" exists, the sketch just doesn't know about it.

So we add an "extern" to the main sketch:

Code: [Select]

extern const int data [] [2];

void setup ()
  {
  Serial.begin (115200);
  Serial.println (data [0] [0]); 
  }  // end of setup
 
void loop ()
  {
   
  }  // end of loop


Still an error:

Code: [Select]

sketch_nov22a.cpp.o: In function `setup':
sketch_nov22a.cpp:10: undefined reference to `data'
sketch_nov22a.cpp:10: undefined reference to `data'


Different error (a linker error, not a compiler error).

The problem is that "const" data is not exported. So remove the word "const" from both files, and it compiles.

But now we ask ourselves, "how big is data"? Well if it was declared in the same file, we can find that out with sizeof. A helpful macro does that:

Code: [Select]

extern int data [] [2];

// number of items in an array
#define NUMITEMS(arg) ((unsigned int) (sizeof (arg) / sizeof (arg [0])))

void setup ()
  {
  Serial.begin (115200);
 
  for (int i = 0; i < NUMITEMS (data); i++)
    Serial.println (data [i] [0]); 
  }  // end of setup
 
void loop () { }


Unfortunately the compiler doesn't know how big data is, because it is in a different file, hence:

Code: [Select]

sketch_nov22a.cpp: In function 'void setup()':
sketch_nov22a:10: error: invalid application of 'sizeof' to incomplete type 'int [][2]'


If we make the extern an explicit size it will compile:

Code: [Select]

extern int data [3] [2];

// number of items in an array
#define NUMITEMS(arg) ((unsigned int) (sizeof (arg) / sizeof (arg [0])))

void setup ()
  {
  Serial.begin (115200);
 
  for (int i = 0; i < NUMITEMS (data); i++)
    Serial.println (data [i] [0]); 
  }  // end of setup
 
void loop () { }


But we have to keep the "extern" in step with the actual variable size in the other (.cpp) file.
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

Nick Gammon

@dhenry: In my particular example, I think the use of the .h file is OK. It works around the issue I described (not knowing the variable size) and in this case the include files were only included once. Plus, I won't be firing myself. ;)
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

dhenry

So here is how it works in your case:

numbers2.h (you can name it whatever you want, as long as it is consistent)
Code: [Select]

//hearder file for data[]
//to be included in user code

extern const int data[] [2];


numbers2.c
Code: [Select]

#include "numbers2.h"

const int data[] [2]={
    {1,1},
  {-2,4},
  {3,-9},
  {4,5}, // Second element here will be 0, Like:  {4,0},
  {-5,-25}
};


Your user code: a simple reference to data[]

Code: [Select]

#include "numbers2.h"

void setup(void) {
}

void loop(void) {
  unsigned char tmp;
 
  //an example of invoking data[];
  tmp+=data[0][(tmp & 0x01)?1:0];
}


numbers2.h can be referenced as many times as you want.

This is fairly standard C.


Nick Gammon

You haven't addressed the question: how do you know how big the array is?
Please post technical questions on the forum, not by personal message. Thanks!

More info:
http://www.gammon.com.au/electronics

Go Up