<Solved> Buffer Overflow, how to debug, how to fix

Greetings fellow members. I have a fully working project, which is already built in on my motorcycle. This unit behaves as a telemetric device for this particular motorcycle, reading sensor data and showing that on a TFT screen.

There is an issue of some variables changed or some functions are called randomly with wrong (impossible) values. I think, this is an issue of buffer overflow and thus arduino behaving randomly weird.

Full Project:

Most common seen problems:
DisplayOnTFT.h - line 27 gets called randomly with a wrong value, even it should be called only once in setup phase.
DisplayOnTFT.h - line 19 the value changes randomly.

I found in an old topic, that you should let “more place in strings before 0”. But who knows what that cryptic old egyptian myth supposed to mean.

Since this problem doesn’t occur deterministically, I have no idea how to debug that. And how to fix it when I found the problem.

Thank you for your help.

Please post your program here.

See How to get the best out of the Forum

...R

Robin2:
Please post your program here.

See How to get the best out of the Forum

...R

Are you kidding mate?

I found in an old topic, that you should let "more place in strings before 0". But who knows what that cryptic old egyptian myth supposed to mean.

If you are referring to the need for an array of chars used to hold a string of characters to have one more element than the number of characters it is not a myth as the array needs to have room for the terminating zero that turns the array of chars into a C style string. Such a string (lowercase s) can then be used by the plethora of C string manipulation functions as well as being printed. So, are your arrays sized appropriately ?

I suspect that you read out of array boundaries or
you get heap fragmentation due to String usage which might cause random restarts, indicating as
"line 27 gets called randomly with a wrong value, even it should be called only once in setup phase."

dr-o:
Are you kidding mate?

No, I was not kidding.

It's much easier to help if we don't have to go to another website to see the stuff you want help with.

Help us to help you.

...R

This appears to be a mistake:

    void applyWriteCommand(String text, int x, int y, int8_t color_r, int8_t color_g, int8_t color_b) {
      char _text[sizeof(text)];
      text.toCharArray(_text, sizeof(_text));

      TFTscreen.stroke(color_r, color_g, color_b);
      TFTscreen.text(_text, x, y);
    }

The 'sizeof' operator will get you the number of bytes in a String object, not the number of characters in the string. Try:

    void applyWriteCommand(String text, int x, int y, int8_t color_r, int8_t color_g, int8_t color_b) {
      char _text[text.length() + 1];
      text.toCharArray(_text, text.length()+1);
      _text[text.length()] = '\0'; // In case 'toCharArray' doesn't add the null terminator.

      TFTscreen.stroke(color_r, color_g, color_b);
      TFTscreen.text(_text, x, y);
    }

Hey Bob, appreciate the explanation of the idea on other forum post. Together with the keyword from noiasca - heap fragmentation -, I can finally fully understand the problem. Tbh, I was not aware of such a problem. Low level programming is really different and has a much more diverse challenges.

@Johnwasser: appreciate it mate. I will check that out and try it. That would really make sense, since the only Problem I can see is on the display. And applyWriteCommand is a very elemental function.

Appreciate it.

This means. From my understanding, functions like

    void updateFuelValue(int oldFuelValue, int fuelValue) {
      removeText(String(oldFuelValue) + "%", TEXTMARGINLEFT , POS1 + TEXTMARGINTOP);
      addText(String(fuelValue) + "%" , TEXTMARGINLEFT , POS1 + TEXTMARGINTOP);
    }

or much “worse”

    double getOilTemperature() {
      return convertOilTemperatureResistanceToCelcius(getOilTemperatureResistance());
    }

cause heap fragmentation and could also make the software more unstable?

@Robin2: Maybe your argument would be valid, if it wasn’t a large Project, and it wasn’t a multi file project. And GitHub code Reviewer is superior to this forums code snippet. Thank you for your contribution nonetheless.

    double getOilTemperature() {
      return convertOilTemperatureResistanceToCelcius(getOilTemperatureResistance());
    }

There is nothing in here that would cause heap fragmentation.

Since only one file from the dozen or so that are on the Github page, is the subject of your question, it would be normal to post it inline as customary, while the Github link is also an informative supplement. You haven't really given any reason for not doing that other than, essentially, "meh".

aarg:

    double getOilTemperature() {

return convertOilTemperatureResistanceToCelcius(getOilTemperatureResistance());
    }




There is nothing in here that would cause heap fragmentation.

Is this so? As much as I understand, function parameters are also saved into memory and destroyed at the end of every loop, which might cause heap fragmentation? Since I don't save the return value of getOilTemperatureResistance into a variable, same for convertOilTemperatureResistanceToCelcius I just wonder, if that wouldn't cause any heap fragmentation. Global variables does. So why not "non saved function return values"?

aarg:
Since only one file from the dozen or so that are on the Github page, is the subject of your question

This is something you claim. I don't claim that. Before I wrote the issue, I didn't even knew, what heap fragmentation is, and before johnwasser gave me a proper solution, I didn't even knew, that .length for string objects existed.

I can assure you, if I knew, where the problem occurs, I would send only the 3-4 lines of code, instead of a whole repo :wink:

Function parameters of basic types only take up space on the stack for a while. Heap fragmentation happens when you're creating and destroying objects like Strings.

dr-o: worth reading

I like this picture ^^:

wildbill:
Function parameters of basic types only take up space on the stack for a while. Heap fragmentation happens when you're creating and destroying objects like Strings.

Which means, casting int values into String with String() function is a potential cause of heap fragmentation, I would guess? In this case, I can only imagine, String is created on heap mem & only the pointer is saved on stack?

dr-o:
Which means, casting int values into String with String() function is a potential cause of heap fragmentation, I would guess?

Yes indeed. If you have plenty of RAM and aren't running your system 24x7 you can get away with using String objects. Probably. On a little Arduino though, Strings are a menace.

noiasca:
dr-o: worth reading

You know you have a memory problem when... | Memories of an Arduino | Adafruit Learning System

I like this picture ^^:

This one is great. I will build in my main loop() a freeMemory() serial Print and let arduino do his thing for half an hour (together with jphnwasser his fix. Lets see, if there are any anomalies. Appreciate the help and new acquired knowledge about Arduino memory. I knew a lot about ProgMem side of the things (have to, if you want to place more data than 32Kb into a Mega 2560). But SRAM was never important to me ^^

@wildbill: depending on freeMemory Result, I will probably give these strings a fix text size. But these text should be almost all same size (maximum 3 chars).

myString.reserve(3);

should do the trick.

It stays stable at 6995 Bytes freeMemory(). I am fine with that. I am going to remove freeMemory serialPrints and will do a live test.

Solved. Well at least, I went out for a ride over an hour and no glitches happened. Since glitches occur non-deterministic, it can also be luck, sure. But that would be a lot of luck.

Really appreciate the new knowledge and especially the help from Johnwasser!