Large ammount of Strings

Hello, im trying to make a solution where i need to Store a large ammount of strings and ID´s

like this: 6565 | Intake Air Temperature Circuit Low Input 7554 | Secondary Upstream Manifold Air Pressure Sensor too low at Idle Engine Running goes on...

Some details: Its 245 Strings with 125 chars(max) each String has an ID

I tried using a function with a switch like this...

switch (idtc) {
                case 3234:
                {
                    return "Description bla bla bla";
                }

well... it says its using 185% of memory... so cant be this way.... After research i found i must use PROGMEM (Flash Memory)

how can i acomplish this? I need to store all 245 Strings with IDs and on my program i must get Description based on ID Do i have to use Struct?

struct DTCST
{
  uint16_t DTCn;
  char* DTCdesc;
};

Anyone can help me? Thanks a lot.

it says its using 185% of memory…

If you’re already at 185% of flash memory, no amount of using PROGMEM is going to help.

I think the OP referred to RAM usage.

const char msg001[] PROGMEM = "message one";
const char msg002[] PROGMEM = "message two";
const char msg003[] PROGMEM = "message three";
const char msg004[] PROGMEM = "message four";

const char* const msgs[] PROGMEM = { msg001, msg002, msg003, msg004, };
const uint16_t values[] PROGMEM = { 0x1000, 0x0100, 0x0010, 0x0001 };

const __FlashStringHelper* findVal(uint16_t what) {
  for (byte i = 0; i < sizeof(values) / sizeof(values[0]); i++) {
    if (pgm_read_word(&values[i]) == what) {
      return (__FlashStringHelper*)pgm_read_word(&msgs[i]);
    }
  }
  return F("not found");
}

void setup() {
  Serial.begin(250000);
  for (byte i = 0; i <= sizeof(values) / sizeof(values[0]); i++) {
    uint16_t val = pgm_read_word(&values[i]);
    Serial.print(F("value 0x"));
    Serial.print(val, HEX);
    Serial.print(F(" maps to \""));
    Serial.print(findVal(val));
    Serial.println(F("\""));
  }
}

void loop() {
}
value 0x1000 maps to "message one"
value 0x100 maps to "message two"
value 0x10 maps to "message three"
value 0x1 maps to "message four"
value 0xDB00 maps to "not found"

I let the array index deliberately go one to high to show the “not found” message.

Storing and Retrieving Strings in the Program Space

You need to post your program, or (better still) a short program that illustrates the problem.

When people use "String" (capital S) they usually mean the String class - but maybe you don't mean that. It is not a good idea to use the String (capital S) class on an Arduino as it can cause memory corruption in the small memory on an Arduino. Just use cstrings - char arrays terminated with 0.

If you just need space for fixed messages such as "Description bla bla bla" you can use the F() macro to store them in PROGMEM.

...R

how can i acomplish this?

You could also use an "Arduino" (as defined by a device which is supported by the Arduino IDE) which can handle all that. Look at an ESP8266 breakout for example.

If you have only 245 strings, why do you use 4-digit IDs?

Each string is x bytes, equaling the string length or number of characters including spaces and punctuation.

If they all 245 strings and IDs are in your program at the time of compile, it's likely they are stored in program memory, which on the Arduino Uno is 32000 to 32768 bytes, which if all you did was store 245 strings is 133-134 bytes per string, but you are also using 4 byte addresses, unless you are using hex IDs or "word"s, in which case, that's 2 bytes per ID, so 129-132 bytes per string...

"Secondary Upstream Manifold Air Pressure Sensor too low at Idle Engine Running" is only 78 bytes, so it's likely that 129-132 sounds like a lot of space, but it's also likely that you have complex coding to manipulate these strings, consuming any and all available program memory.

PROGMEM is useful to manage limited RAM at the cost of read/write speed. Also, PROGMEM has a limited number of writes, very large number, but still limited.

Since you have so much program, it's likely you have very little to no PROGMEM left to write to and read from.

Whandall: Thanks for your help, i´ve seen that "solution", but kinda dont like it, because i have to declare 245 strings both msgs[] and values[] synced. I would prefer another "indexed" solution something like multi dimentional array or struct. Also i was trying to avoid using loops to search my correct String. but again, i apreciate your post :)

Robin2: Yes i always avoid using Strings, sorry im C# guy, old habits, hehehe here in arduino i always use char and *char Ye, i guess im going to Try that F() function to store my "strings"(hehe) in PORGMEM

Perehama: ahaha , really nice post. Your right about IDs , actually there are not "ID´s", they are DTC´s Codes (BMW specific-4bytes) Im Building a custom Gauge, Real Data with Oficial BMW CAN Addresses (already Done), Diagnostic(BMW DTC´s) missing only DTC description, where i need to store 245 DTC description, and Datalog witch is next thing todo.

Im using Arduino MEGA 2560 because i need several Serials and its the "best" arduino for this job(at least i think) Im getting 200ms each reading from BMW ECU (Oficial CAN Address, not generic PIDs) :)

so.... i Have : DTC_CODE(uint16_t) & DTC_DESCRIPTION(125bytes max)

my actual code:

char *GetDTC(int idtc){ switch (idtc) { case 3464: { return "AISIN TCU OBD relevant fault"; } case 6334: { return "Air intake temperature and Air ambient temperature sensors Implausible value"; } } }

All i need is to Store 245 DTC_Description(125char) into arduino and search for a specific code like

DTCdesc = GetDTC(6334);

What you think?

I love SPI. There are other buses, but SPI makes sense to me. If you aren't using your SPI bus for GPIO and you have one additional pin, for "slave select" including the Slave Select pin, consider external flash. Open source flash like the one linked have libraries and code examples to learn how to use them.

In practice, I’d load the IDs and messages into the flash using a utility sketch to do just that (and a PC serial port and copy/paste if I was doing it).

Then, I’d attach the flash to my project and only have code to read the data, saving a lot of program memory.

Perehama Thats a very good suggestion actually, i didnt knew about that Flash :) seems really cool. witch one? https://www.adafruit.com/product/1897 or https://www.adafruit.com/product/1895 ?

i´ve seen that PROGMEM can be written/cycle 10.000 times that means everytime i turn on arduino it writes to Progmem and counts as 1 (each string) ? is that it? or only counts only first time i write code to arduino?

x4code: Perehama Thats a very good suggestion actually, i didnt knew about that Flash :) seems really cool. witch one? https://www.adafruit.com/product/1897 or https://www.adafruit.com/product/1895 ?

i´ve seen that PROGMEM can be written/cycle 10.000 times that means everytime i turn on arduino it writes to Progmem and counts as 1 (each string) ? is that it? or only counts only first time i write code to arduino?

1897 is SPI bus and 1895 is I2C. It is a matter of preference. An SPI bus uses 3 wires, SCK, MOSI, and MISO for the bus and one pin SS per slave on the bus, and is theoretically faster given that it takes 1 bit to send 1 bit. I2C uses 2 wires, but uses a message address so sending 1 bit means sending a data packet containing that bit.

Each write to an address space in PROGMEM can be done X times. Each time you turn it on, and it writes x to address y in memory, both the device ID bits and the message bits, it can do that X times to that location in memory.

If you are writing to PROGMEM data that is read from an array in your sketch, you are reading from program memory and writing to program memory....

The unanswered question is where are you storing the device IDs and messages.

If They are stored on device x connected to your Arduino, and you simply read and manipulate the values, they aren't a factor in program memory until you write them via PROGMEM, but the assumption is that they are part of your program at compile.

Perehama: Each write to an address space in PROGMEM can be done X times. Each time you turn it on, and it writes x to address y in memory, both the device ID bits and the message bits, it can do that X times to that location in memory.

If you are writing to PROGMEM data that is read from an array in your sketch, you are reading from program memory and writing to program memory....

Normally only the bootloader writes to PROGMEM, or its done via ICSP.

Only reading data from system flash does not wear it out.

Whandall:
Normally only the bootloader writes to PROGMEM, or its done via ICSP.

Only reading data from system flash does not wear it out.

Yes, to understand the difference between flash, SRAM and EEPROM, http://playground.arduino.cc/Learning/Memory
PROGMEM is the exception to normal, see https://www.arduino.cc/en/Reference/PROGMEM
One can also write to EEPROM using the EEPROM library, see Arduino - EEPROM

If you have read that, why are you writing confusing stuff?

Perehama: Each time you turn it on, and it writes x to address y in memory, both the device ID bits and the message bits, it can do that X times to that location in memory.

There is no limit to RAM read/writes.

Perehama: Each time you turn it on, and it writes x to If you are writing to PROGMEM data that is read from an array in your sketch, you are reading from program memory and writing to program memory....

that only makes kind of sense if you are writing to PROGMEM.

Why are you taking part of my message out of context. It's obvious that I am talking about PROGMEM and flash.

Perehama: It's obvious that I am talking about PROGMEM and flash.

Given that

Each time you turn it on, and it writes x to address y in memory, both the device ID bits and the message bits, it can do that X times to that location in memory.

is plain wrong.

It (the Flash or PROGMEM Flash?) does nothing when it is turned on. It is not capable of writing to any memory.

x4code:
i´ve seen that PROGMEM can be written/cycle 10.000 times
that means everytime i turn on arduino it writes to Progmem and counts as 1 (each string) ? is that it?
or only counts only first time i write code to arduino?

You write to PROGMEM when you upload your program, not when you start the Arduino.

As you’re asking the above, you don’t currently have the knowledge (no offense intended) to write PROGMEM from the uploaded code. So don’t worry about those 10,000 cycles.

i got an idea, what about i define all my DTC´s like this...

const char P1791[] PROGMEM = "AISIN TCU OBD relevant fault"; const char P1792[] PROGMEM = "AISIN TCU OBD relevant fault222"; const char P1793[] PROGMEM = "AISIN TCU OBD relevant fault3333";

and on my main code i get the DTCcode witch for example is 1792, how can i read that P1791[] stored in Progmem??? Any ideas?? good solution?

x4code: i got an idea, what about i define all my DTC´s like this...

const char P1791[] PROGMEM = "AISIN TCU OBD relevant fault"; const char P1792[] PROGMEM = "AISIN TCU OBD relevant fault222"; const char P1793[] PROGMEM = "AISIN TCU OBD relevant fault3333";

and on my main code i get the DTCcode witch for example is 1792, how can i read that P1791[] stored in Progmem???

How would you read the data from RAM?

The names of all variables are gone after compiling.

x4code: good solution?

If you change the compiler to support such operations, a good solution, otherwise a dream.

x4code:
i got an idea, what about i define all my DTC´s like this…

const char P1791 PROGMEM = “AISIN TCU OBD relevant fault”;
const char P1792 PROGMEM = “AISIN TCU OBD relevant fault222”;
const char P1793 PROGMEM = “AISIN TCU OBD relevant fault3333”;

and on my main code i get the DTCcode witch for example is 1792,
how can i read that P1791 stored in Progmem???
Any ideas?? good solution?

Yes, you could write a macro to do that.