Memory overflow

Hello Comunity,

I have a problem with a project of mine. Here is the project.

I have points of interests. When I am near these points of interests, I get information about this points of interest. There are over 6000 points of interests. A point of interest is an array itself with three fields.

So I created a huge array

const float pointsOfInterest[][3] PROGMEM = {
...
};

When I tried to compile that, java crashed and got the error: Too long array.

So I seperated the array in three parts and compiler doesn't make any problems anymore. The arrays are pointsOfInterest1,2,3 - each one having almost 2000 POIs. When I try to read from the memory via

pgm_read_float(&pointsOfInterest1[firstDimension][secondDimension]);

It works. Everything works. TFT works, input and outputs work. PWM works, A-D Convertion works.
This is because the compiler only writes one array into the progmem (unused variables are not written).

When I read all three, 88kb Data got written on PROGMEM and after upload finished, nothing works. TFT doesn't work. Serial.print doesn't work. Can't debug.

So what I need is something like a garbage collector. Can't find anything to that though. Or I just say, forget progmem and reading out of it. Just install more ram. Is there any way to increase the ram?

I have a mega with 256kb progmem and 8kb sram.

Thank you for your help.

dr-o:
When I read all three, 88kb Data [...]

I have a mega with [...] and 8kb sram.

You have 8kB of SRAM and are trying to read 88kB - this will not fit. Why are you reading all POI's at once?

Am I? I thought I am reading only one information from progmem by using

pgm_read_float(&pointsOfInterest1[firstDimension][secondDimension]);

How can I read only one information at a time from PROGMEM?

What I am trying to do is every 5 minutes, it goes through all POIs and creates a small array including every POI in a certain radius - which will be in ram. Then it is much faster to find the nearest POI or using traveling salesman algorithm etc. etc.

dr-o:
Am I? I thought I am reading only one information from progmem by using

Ah, I thought you were reading all three arrays. Your approach is correct, you are reading only single floats.
How small is your "small array"? Can you show the complete Code? Are you using String (capital S)?

The small array part is not important at this point. I need to have a read on my POI positions before I can implement the list of nearby POIs.

Here is the Git repository of the Project.

You need the far versions of the pgm_read_ functions:

pgm_read_float_far(pgm_get_far_address(pointsOfInterest1[firstDimension][secondDimension]))
// no & there -------------------------^

oqibidipo:
You need the far versions of the pgm_read_ functions:

pgm_read_float_far(pgm_get_far_address(pointsOfInterest1[firstDimension][secondDimension]))

// no & there -------------------------^

Doesn't change anything. As much as I understand, _far makes it more precise since it doesn't read 16 but 32 bits?

PROGMEM access on a mega with very large arrays works a bit different.

Try this:

float x = pgm_read_float_far(pgm_get_far_address(pointsOfInterest) + firstDimension * sizeof(pointsOfInterest[0]) + secondDimension * sizeof(pointsOfInterest[0][0]));

nothing works. TFT doesn't work. Serial.print doesn't work. Can't debug.

The cause of nothing working is that the pin to register mapping tables in PROGMEM get pushed beyond 64 kilobytes.
You can avoid that this way:

#define PROGMEM_LATE __attribute__((section("progmem.zzz")))

const float radarPositions1[][3] PROGMEM_LATE = { 
...

oqibidipo:
The cause of nothing working is that the pin to register mapping tables in PROGMEM get pushed beyond 64 kilobytes.
You can avoid that this way:

#define PROGMEM_LATE __attribute__((section("progmem.zzz")))

const float radarPositions1[][3] PROGMEM_LATE = {
...

awesome, this works!

Thanks!

You mind explaining what is happening here? What is progmem.zzz? and where is it defined?