Not enough memory

pcbbc:
You have two floats (2x4=8 bytes) and a 5 character string to store.
Am I correct?
And the 5 character string is always uppercase A-Z. No digits, symbols or lowercase?

No, the Strings do also have lowercase and numbers in them, because some points' names are their coordinates, e.g. 5115N. And they're not always 5 characters long, some are only three characters long, otehrs are more...

pcbbc:
How many of these items do you have to store?

In total I have to store about 10'000 navigation point names and their respective coordinates.

If you are thinking of adding external memory, then if you need convenience, maybe because the coordinates change, then an SD card may be a way to go. But there's an overhead in using the SD library.

Alternatively, if the coordinates don't change, then maybe something simpler such as an I2C or SPI flash device with a much lower overhead would be the way to go.

If the coordinate "file" is the only file as it were, then you can ditch a filesystem on the storage device and just access it as raw memory and have more program space on your processor.

If you access the coordinates by name, then presort them into alphabetical order before uploading them to the flash device.

Drop the idea of using an if statement for each Navpoint. Put all the Navpoints into an array then check each element of the array against navpoint until you find a match. You are consuming huge amount of program memory with repetitive code that can be made much smaller.

david_2018:
Drop the idea of using an if statement for each Navpoint. Put all the Navpoints into an array then check each element of the array against navpoint until you find a match. You are consuming huge amount of program memory with repetitive code that can be made much smaller.

For example by using a struct?
Because if so, I wasn't able to find any good tutorials on how to access the data from a struct...

norbyx:
For example by using a struct?
Because if so, I wasn’t able to find any good tutorials on how to access the data from a struct…

Think of it like a class

Look at my example with the struct (reply #12). The major advantage is that you only need a for-loop and one if-statement.

The array size is unfortunately limited to around 1500 navigation points; see reply #18 by david_2018 for the reason (I haven't worked around that one yet). Reason seems to be that anarray is limited to 32 kBytes; no idea why.

Been looking through your data, and notice that there are several instances of duplicate Navpoint names, how are you going to differentiate between them?

These are some of the largest, but there are numerous instance where the same name is used for two navpoints.

const char Navpoint511[]PROGMEM = {"HORTA"};
const char Navpoint3574[]PROGMEM = {"HORTA"};
const char Navpoint3575[]PROGMEM = {"HORTA"};
const char Navpoint6343[]PROGMEM = {"HORTA"};

const char Navpoint3255[]PROGMEM = {"FIR03"};
const char Navpoint3256[]PROGMEM = {"FIR03"};
const char Navpoint3257[]PROGMEM = {"FIR03"};
const char Navpoint3258[]PROGMEM = {"FIR03"};
const char Navpoint6293[]PROGMEM = {"FIR03"};

Just to see where one could get with the program size, I’ve put the below in 00_Navpoints.

// structure to store navigation point
struct NAVPOINT
{
  char name[8];
  float coordis[2];
};

const NAVPOINT Navpoints1[1600] PROGMEM = 
{
};

const NAVPOINT Navpoints2[1600] PROGMEM = 
{
};

const NAVPOINT Navpoints3[1600] PROGMEM = 
{
};

const NAVPOINT Navpoints4[1600] PROGMEM = 
{
};

const NAVPOINT Navpoints5[1600] PROGMEM = 
{
};

const NAVPOINT Navpoints6[1600] PROGMEM = 
{
};

The arrays are empty; I’ll leave it up to you to populate them. It’s good for 9600 names plus coordinates. The arrays are not optimised away; you can see that if you remove one of the for-loops in below.

Next I modified your IFFS code as shown below; it’s a rough hack, but I think that it’s correct.

void IFFS() {
  NAVPOINT tmp;

  if (runonceFPLN == false && initcoordis == true) {
    for (byte i = 0; i <= Flightplanlength - 1; i++) {
      Navpoint = Flightplan[i];
      Navpoint.toCharArray(navpoint, Navpoint.length() + 1);

      bool found = false;

      for (uint16_t navCnt = 0; navCnt < sizeof(Navpoints1) / sizeof(Navpoints1[0]); navCnt++)
      {
        // get navpoint from progmem
        memcpy_P(&tmp, &(Navpoints1[navCnt]), sizeof(NAVPOINT));

        // compare
        if (strcmp(tmp.name, navpoint) == 0)
        {
          found = true;
          break;
        }
      }

      if (found == false)
      {
        for (uint16_t navCnt = 0; navCnt < sizeof(Navpoints2) / sizeof(Navpoints2[0]); navCnt++)
        {
          // get navpoint from progmem
          memcpy_P(&tmp, &(Navpoints2[navCnt]), sizeof(NAVPOINT));

          // compare
          if (strcmp(tmp.name, navpoint) == 0)
          {
            found = true;
            break;
          }
        }
      }

      if (found == false)
      {
        for (uint16_t navCnt = 0; navCnt < sizeof(Navpoints3) / sizeof(Navpoints3[0]); navCnt++)
        {
          // get navpoint from progmem
          memcpy_P(&tmp, &(Navpoints3[navCnt]), sizeof(NAVPOINT));

          // compare
          if (strcmp(tmp.name, navpoint) == 0)
          {
            found = true;
            break;
          }
        }
      }

      if (found == false)
      {
        for (uint16_t navCnt = 0; navCnt < sizeof(Navpoints4) / sizeof(Navpoints3[0]); navCnt++)
        {
          // get navpoint from progmem
          memcpy_P(&tmp, &(Navpoints4[navCnt]), sizeof(NAVPOINT));

          // compare
          if (strcmp(tmp.name, navpoint) == 0)
          {
            found = true;
            break;
          }
        }
      }

      if (found == false)
      {
        for (uint16_t navCnt = 0; navCnt < sizeof(Navpoints5) / sizeof(Navpoints5[0]); navCnt++)
        {
          // get navpoint from progmem
          memcpy_P(&tmp, &(Navpoints5[navCnt]), sizeof(NAVPOINT));

          // compare
          if (strcmp(tmp.name, navpoint) == 0)
          {
            found = true;
            break;
          }
        }
      }

      if (found == false)
      {
        for (uint16_t navCnt = 0; navCnt < sizeof(Navpoints6) / sizeof(Navpoints6[0]); navCnt++)
        {
          // get navpoint from progmem
          memcpy_P(&tmp, &(Navpoints6[navCnt]), sizeof(NAVPOINT));

          // compare
          if (strcmp(tmp.name, navpoint) == 0)
          {
            found = true;
            break;
          }
        }
      }


      if (found == true)
      {
        coordis[0] = tmp.coordis[0];
        coordis[1] = tmp.coordis[1];
      }



      switch (i) {
        case 0:

          break;
      }/*
      if (i >= 1){
        alpha = acos(sin((2 * pi * prevcoordis[0]) / 360) * sin((2 * pi * coordis[0]) / 360) + cos((2 * pi * prevcoordis[0]) / 360) * cos((2 * pi * coordis[0]) / 360) * cos((2 * pi * (prevcoordis[1] - coordis[1])) / 360));
        alpha = (alpha * 360) / (2 * pi);
        //distances[i - 1] = 2 * pi * rE * alpha / 360;
      }*/
      //else{
      prevcoordis[0] = coordis[0];
      prevcoordis[1] = coordis[1];
      // }
    }
  }
}

I hope that I did not cut out too musch of those 12000-plus lines :wink:

Compile result:

Sketch uses 205834 bytes (81%) of program storage space. Maximum is 253952 bytes.
Global variables use 1783 bytes (21%) of dynamic memory, leaving 6409 bytes for local variables. Maximum is 8192 bytes.

I did not look at any other variables that you used !!

Notes:
1)
You can probably use an array (of pointers to the) Navpoint arrays and loop through those; I did not feel like analysing that.
2)
I tested it with some partial data in a sketch based on my previous one in reply #12 (8x1600 navigation points) to see if the principle of finding a navigation point works.

I have my doubts about the code from the previous post, mainly because of the need to use far address references with a mega. This is my version of your IFFS code, looks quite cryptic but PROGMEM can get that way on a mega (and I do not have nearly as much experience as others, so may be doing it the hard way).

void IFFS() {
  if (runonceFPLN == false && initcoordis == true) {
    for (byte i = 0; i <= Flightplanlength - 1; i++) {
      Navpoint = Flightplan[i];
      Navpoint.toCharArray(navpoint, Navpoint.length() + 1);


      //get the base address of each array
      //necessary because the compiler does not handle this properly
      uint_farptr_t baseNavpointsA = pgm_get_far_address(NavpointsA);
      uint_farptr_t baseNavpointsB = pgm_get_far_address(NavpointsB);
      uint_farptr_t baseNavpointsC = pgm_get_far_address(NavpointsC);
      
      //get the number of elements in each of the arrays
      const uint16_t numNavpointsA = sizeof(NavpointsA) / sizeof(NavpointsA[0]);
      const uint16_t numNavpointsB = sizeof(NavpointsB) / sizeof(NavpointsB[0]);
      const uint16_t numNavpointsC = sizeof(NavpointsC) / sizeof(NavpointsC[0]);
      const uint16_t totalNavPoints = numNavpointsA + numNavpointsB + numNavpointsC;

      NavpointStruct NavpointBuff;
      uint_farptr_t testNavpoint;
      bool noMatch = true;
      uint16_t j = 0;
      
      while (noMatch && (j < totalNavPoints )) {
        //calculate the address of the array element
        if (j < numNavpointsA) {
          testNavpoint = baseNavpointsA + j * sizeof(NavpointsA[0]);
        } else if (j < (numNavpointsA + numNavpointsB)) {
          testNavpoint = baseNavpointsB + (j - numNavpointsA) * sizeof(NavpointsB[0]);
        } else {
          testNavpoint = baseNavpointsC + (j - numNavpointsA - numNavpointsB) * sizeof(NavpointsC[0]);
        }

        if (strcmp_PF(navpoint, testNavpoint) == 0) { //testNavpoint points to npText within the array element
          //copy the array element to ram when found, to make it easier to access
          memcpy_PF(&NavpointBuff, testNavpoint, sizeof(NavpointBuff));
          coordis[0] = NavpointBuff.npLat;
          coordis[1] = NavpointBuff.npLong;
          noMatch = false;
        }
        j++;
      }
      //if noMatch == true at this point, then navpoint was not found

      switch (i) {
        case 0:

          break;
      }/*
      if (i >= 1){
        alpha = acos(sin((2 * pi * prevcoordis[0]) / 360) * sin((2 * pi * coordis[0]) / 360) + cos((2 * pi * prevcoordis[0]) / 360) * cos((2 * pi * coordis[0]) / 360) * cos((2 * pi * (prevcoordis[1] - coordis[1])) / 360));
        alpha = (alpha * 360) / (2 * pi);
        //distances[i - 1] = 2 * pi * rE * alpha / 360;
      }*/
      //else{
      prevcoordis[0] = coordis[0];
      prevcoordis[1] = coordis[1];
      // }
    }
  }
}

Also the code where the arrays are defined (complete file attached below, do not rely on the data actually being match up correctly).

//this define tells the compiler to place the PROGMEM data
//  in the upper section of flash memory, if you do not do 
//  this, it will displaces some pre-defines PROGMEM data
//  that must reside in the lower secion of flash
#define PROGMEM_FAR __attribute__((section(".fini7")))

//left these in the code because they are referenced elsewhere in your code
const float PROGMEM LSGG[2] = {46.23763, 6.11010};
const float PROGMEM LPPR[2] = {41.23761, -8.67557};

//define a structure to hold the navpoint and its coordinates
struct NavpointStruct {
  char npText[6];
  float npLat;
  float npLong;
};

//define the actual data tables
//limited to 2340 elements because an array can not exceed 32767 bytes
const NavpointStruct NavpointsA[2340] PROGMEM_FAR = {
  {"ARBOL", 43.516945, 70.854446},
  {"ARDAG", 51.284443, 20.69389},
  {"ARDOK", 43.404999, 34.105},
  {"ARGES", 44.082222, 26.826666},

00_Navpoints.ino (215 KB)

david_2018:
I have my doubts about the code from the previous post, mainly because of the need to use far address references with a mega.

I understand the concern; testing showed that it worked. But it might not be the right way and possibly I was just lucky.

There are some things in your code that I did not know about; thanks for that, more study to do :wink:

Did you compile the complete code with your modifications? If so, how far did you bring the program memory down?

sterretje:
I understand the concern; testing showed that it worked. But it might not be the right way and possibly I was just lucky.

There are some things in your code that I did not know about; thanks for that, more study to do :wink:

Did you compile the complete code with your modifications? If so, how far did you bring the program memory down?

I did compile the entire code, using the data in the original 00_Navpoints.ino file for the arrays, memory usage will go up with more Navpoints.

Sketch uses 144638 bytes (56%) of program storage space. Maximum is 253952 bytes.
Global variables use 1766 bytes (21%) of dynamic memory, leaving 6426 bytes for local variables. Maximum is 8192 bytes.

I did a bit of testing using the serial monitor, but don't have the correct display to do any extensive tests, and have not looked through the all the code to try and understand it.

Great. So the point that is made is that OP's code can be reworked to fit in the program memory.

david_2018:
I have my doubts about the code from the previous post, mainly because of the need to use far address references with a mega. This is my version of your IFFS code, looks quite cryptic but PROGMEM can get that way on a mega (and I do not have nearly as much experience as others, so may be doing it the hard way).

void IFFS() {

if (runonceFPLN == false && initcoordis == true) {
   for (byte i = 0; i <= Flightplanlength - 1; i++) {
     Navpoint = Flightplan[i];
     Navpoint.toCharArray(navpoint, Navpoint.length() + 1);

//get the base address of each array
     //necessary because the compiler does not handle this properly
     uint_farptr_t baseNavpointsA = pgm_get_far_address(NavpointsA);
     uint_farptr_t baseNavpointsB = pgm_get_far_address(NavpointsB);
     uint_farptr_t baseNavpointsC = pgm_get_far_address(NavpointsC);
     
     //get the number of elements in each of the arrays
     const uint16_t numNavpointsA = sizeof(NavpointsA) / sizeof(NavpointsA[0]);
     const uint16_t numNavpointsB = sizeof(NavpointsB) / sizeof(NavpointsB[0]);
     const uint16_t numNavpointsC = sizeof(NavpointsC) / sizeof(NavpointsC[0]);
     const uint16_t totalNavPoints = numNavpointsA + numNavpointsB + numNavpointsC;

NavpointStruct NavpointBuff;
     uint_farptr_t testNavpoint;
     bool noMatch = true;
     uint16_t j = 0;
     
     while (noMatch && (j < totalNavPoints )) {
       //calculate the address of the array element
       if (j < numNavpointsA) {
         testNavpoint = baseNavpointsA + j * sizeof(NavpointsA[0]);
       } else if (j < (numNavpointsA + numNavpointsB)) {
         testNavpoint = baseNavpointsB + (j - numNavpointsA) * sizeof(NavpointsB[0]);
       } else {
         testNavpoint = baseNavpointsC + (j - numNavpointsA - numNavpointsB) * sizeof(NavpointsC[0]);
       }

if (strcmp_PF(navpoint, testNavpoint) == 0) { //testNavpoint points to npText within the array element
         //copy the array element to ram when found, to make it easier to access
         memcpy_PF(&NavpointBuff, testNavpoint, sizeof(NavpointBuff));
         coordis[0] = NavpointBuff.npLat;
         coordis[1] = NavpointBuff.npLong;
         noMatch = false;
       }
       j++;
     }
     //if noMatch == true at this point, then navpoint was not found

switch (i) {
       case 0:

break;
     }/*
     if (i >= 1){
       alpha = acos(sin((2 * pi * prevcoordis[0]) / 360) * sin((2 * pi * coordis[0]) / 360) + cos((2 * pi * prevcoordis[0]) / 360) * cos((2 * pi * coordis[0]) / 360) * cos((2 * pi * (prevcoordis[1] - coordis[1])) / 360));
       alpha = (alpha * 360) / (2 * pi);
       //distances[i - 1] = 2 * pi * rE * alpha / 360;
     }*/
     //else{
     prevcoordis[0] = coordis[0];
     prevcoordis[1] = coordis[1];
     // }
   }
 }
}




Also the code where the arrays are defined (complete file attached below, do not rely on the data actually being match up correctly).



//this define tells the compiler to place the PROGMEM data
//  in the upper section of flash memory, if you do not do
//  this, it will displaces some pre-defines PROGMEM data
//  that must reside in the lower secion of flash
#define PROGMEM_FAR attribute((section(".fini7")))

//left these in the code because they are referenced elsewhere in your code
const float PROGMEM LSGG[2] = {46.23763, 6.11010};
const float PROGMEM LPPR[2] = {41.23761, -8.67557};

//define a structure to hold the navpoint and its coordinates
struct NavpointStruct {
 char npText[6];
 float npLat;
 float npLong;
};

//define the actual data tables
//limited to 2340 elements because an array can not exceed 32767 bytes
const NavpointStruct NavpointsA[2340] PROGMEM_FAR = {
 {“ARBOL”, 43.516945, 70.854446},
 {“ARDAG”, 51.284443, 20.69389},
 {“ARDOK”, 43.404999, 34.105},
 {“ARGES”, 44.082222, 26.826666},

I’ve been successful with this variant. Thank you very much to everyone for helping me so far!! :smiley:
I added all the remaining waypoints and now I’m back up au 73% program space usage…
I’m afraid that I won’t have enough memory for the rest of the project.
I’ve been looking for other means to expand the memory of the Arduino: adding an EEPROM IC with I2C, and adding an SD card reader…

I don’t know which one is the “smartest” one to use because I only have 5 Pins left to use on the Arduino Mega. All the other ones are either being used for the button matrix or for the UTFT display shield.

Cockpit_Arduino_MCDU.zip (126 KB)

You are actually up to about 85% program memory usage, you increased the length of the text without increasing the store allocated for it. You could save a bit of memory by compressing the text, but not a tremendous amount.

What type display/shield are you using that needs that many pins on a mega?

david_2018:
What type display/shield are you using that needs that many pins on a mega?

I’m using a “TFT LCD Mega Shield” (TFT LCD Mega Shield für Arduino Mega 2560, 5,24 €)
and a “5,0” 800x480 TFT LCD Display" (https://eckstein-shop.de/50-800x480-TFT-LCD-Display-ohne-Touchscreen-MD050SD-MCU-Arduino-Kompatibel)

I couldn’t find the datasheet of either…

And I have around 70 buttons so, 23 Pins for the matrix (using all the analog pins except for one)

Schematic of the shield:

TFT_MEGA_V2.0.pdf (41.6 KB)

What else do you have to add to the code? The code itself is not huge, taking up a little over 18% of the memory, with the navpoint arrays and the font tables taking up the rest. Some of the code is quite repetitive, and could be made a bit more efficient if needed. Changing all the String variables to char arrays usually saves a few thousand bytes of memory also, and would generally be a good idea anyway.

Are there any other changes you are going to be making to the navpoint data? The text looks like it could be compressed quite a bit, but I'm not going to spend much time looking at that if its not in the final form yet.

david_2018:
What else do you have to add to the code? The code itself is not huge, taking up a little over 18% of the memory, with the navpoint arrays and the font tables taking up the rest. Some of the code is quite repetitive, and could be made a bit more efficient if needed. Changing all the String variables to char arrays usually saves a few thousand bytes of memory also, and would generally be a good idea anyway.

Are there any other changes you are going to be making to the navpoint data? The text looks like it could be compressed quite a bit, but I'm not going to spend much time looking at that if its not in the final form yet.

I wanted to add a function in which if there are multiple waypoints with the same name, one could choose which one would be the right one
In the end it should be as close as possible to the things this MCDU emulator does (https://www.mcdu.equicom.net/)

Interesting webpage on the MCDU emulator, explains a lot about what you are doing with the arduino. Do you intend to store a database of airport runway information? If so, then you will have to have some additional storage.

As far as having multiple waypoints with the same name, I have not found a way to actually enter a waypoint on the emulator, so cannot see how it handles duplicate names, but as long as you can find a way to display the choices it shouldn't be much of a problem, at most you will have a list of six to choose from (for FIR04). If you really wanted to, the program could calculate the distance from the previous waypoint and sort the list from nearest to farthest away.

I’ve now tried to hook up an SD card reader. It works fine but as soon as I put the UTFT shield on the Arduino (without connecting the display to the shield) all of a sudden it stops working and it says: “SD card initialization failed”
I already tried to change the SS (or CS) pin for it but it still doesnt work.
Now I noticed that the display itself has its own SD card reader and I’ve seen tutorials of people reading bitmaps from it online. Does someone know if it’s also possible to read .txt files, for data, from that SD slot?

Edit: I used the ICSP pins for the ISP communications

This is the code I used to determine if the reader works:

/*
 *  Arduino SD Card Tutorial Example
 *  
 *  by Dejan Nedelkovski, www.HowToMechatronics.com
 */
#include <SD.h>
#include <SPI.h>
File myFile;
int pinCS = A15; // Pin 10 on Arduino Uno
void setup() {
    
  Serial.begin(9600);
  pinMode(pinCS, OUTPUT);
  
  // SD Card Initialization
  if (SD.begin())
  {
    Serial.println("SD card is ready to use.");
  } else
  {
    Serial.println("SD card initialization failed");
    return;
  }
  
  // Create/Open file 
  myFile = SD.open("test.txt", FILE_WRITE);
  
  // if the file opened okay, write to it:
  if (myFile) {
    Serial.println("Writing to file...");
    // Write to file
    myFile.println("Testing text 1, 2 ,3...");
    myFile.close(); // close the file
    Serial.println("Done.");
  }
  // if the file didn't open, print an error:
  else {
    Serial.println("error opening test.txt");
  }
  // Reading the file
  myFile = SD.open("test.txt");
  if (myFile) {
    Serial.println("Read:");
    // Reading the whole file
    while (myFile.available()) {
      Serial.write(myFile.read());
   }
    myFile.close();
  }
  else {
    Serial.println("error opening test.txt");
  }
  
}
void loop() {
  // empty
}
int pinCS = A15; // Pin 10 on Arduino Uno

One of those statements is incorrect