SDFat crashes Arduino???

Here is the subroutine:

void run_test()
 {
  static char log_filename[64];
  char RTC_Time_now[20];
  
Serial.println("1");
  if (Run_Status == START)
   {
Serial.println("START");
    Reset_Counts();
    Run_Status = RUNNING;
    get_time_date_string(RTC_Time_now);
    sprintf(log_filename, "%s %dC %dmA-Hr.log", RTC_Time_now, voltage_sel, (current_sel*100));
Serial.print("log_filename: ");    
Serial.println(log_filename);
    if (!file.open(log_filename, O_CREAT | O_WRITE | O_EXCL))
     {
      Serial.println(F("File open error"));
     }
     else
      {
      Serial.println(F("File open success"));
      }
   }
Serial.println("2");
  if (Run_Status == RUNNING)
   {
    Serial.println("RUNNING");
   }

  // test for end condition
Serial.println("3");
  if (((Run_Status == RUNNING) && (battery_voltage < (voltage_sel *100))) || (Run_Status == STOP))
   {
    Serial.println("END TEST");
    if (Run_Status == STOP)
     { Run_Status = STOPPED; }
     else
      { Run_Status = COMPLETE; }
    file.close();  
   }
  Serial.println("Exit run_test()"); 
 }

And the debug output

1
2
3
Exit run_test()
1
START
2018-12-18 22:16:47
log_filename: 2018-12-18 22-16-47 1C 100mA-Hr.log
18 22:16:47
log_

Three items I note.

  1. It is an UNO, and the program is compiling with the following output:
    Sketch uses 20498 bytes (63%) of program storage space. Maximum is 32256 bytes.
    Global variables use 1732 bytes (84%) of dynamic memory, leaving 316 bytes for local variables. Maximum is 2048 bytes.
    Low memory available, stability problems may occur.

  2. Once it crashes it spews garbage to the serial monitor

  3. It is happening around the file.open command. I am using the SDFat library. I hope. I have SdFat.h as the include.

Does the SDFat library use any of the timers? This program does use the TIMER2 interrupt.

Once it reaches this point, the program has crashed. Nothing is responding (serial, I/O, LCD). How low can I go on the memory? It is showing a warning to that effect, but how low can I go?

I don’t see anything obviously wrong. Anyone else have any thoughts?

P.S. I realize it is a code snippet. The program doesn’t fit in code tags and I have isolated it to this section.

The compiler gave you a HUGE hint:

Low memory available, stability problems may occur.

You have some Serial prints that are printing constant strings that aren't using the F macro. That would be a great place to start saving some RAM.

Serial.println("END TEST");

becomes

Serial.println(F("END TEST"));

and that saves 9 bytes.

How low can I go on the memory? It is showing a warning to that effect, but how low can I go?

The memory report only shows some types of memory usage and not others. For example the buffers for WS2812 LEDs don’t seem to show up. Also memory is needed for things like return stacks so it depends on how deep your function calls are. Things like String ( not string , case sensitive here ) can cause problems due to lack of garbage collection.

I would try the whole thing on a Mega or Leonardo or Micro as they have more memory to see if the memory is your problem.

Other things that save memory is using byte variables when possible instead of always using int, using #defines instead of named constants and reducing the depth of any function calls. Also look at the size of any array and see if that could be encoded more efficiently .

if you have changed the boards.txt file (= increased the serial buffers) it is time to think reseting to lower values.

Grumpy_Mike:
using #defines instead of named constants

Usually the compiler is smart enough to embed all constant expressions in the code without using SRAM. You don't even have to use the const keyword.

Getting "clever" with #define often has the opposite effect as it prevents the optimization from working fully.

Delta_G:
The compiler gave you a HUGE hint:

You have some Serial prints that are printing constant strings that aren't using the F macro. That would be a great place to start saving some RAM.

Serial.println("END TEST");

becomes

Serial.println(F("END TEST"));

and that saves 9 bytes.

Only those serial prints that are for debugging are not using the F macro. All other serial print statements of constant value use the F macro.

Grumpy_Mike:
The memory report only shows some types of memory usage and not others. For example the buffers for WS2812 LEDs don’t seem to show up. Also memory is needed for things like return stacks so it depends on how deep your function calls are. Things like String ( not string , case sensitive here ) can cause problems due to lack of garbage collection.

I would try the whole thing on a Mega or Leonardo or Micro as they have more memory to see if the memory is your problem.

Other things that save memory is using byte variables when possible instead of always using int, using #defines instead of named constants and reducing the depth of any function calls. Also look at the size of any array and see if that could be encoded more efficiently .

No Strings. No variables of a type larger than they need to be (if a loop runs 0 to 7, then x is a byte). Non changing values are #define(d) not assigned to variables that never change value. I have one array and it is for the filename (coincidentally listed in the snippet with the problem).

Now that we have effectively covered the basics of defining variable for a microcontroller (ie., a microprocessor with limited RAM). Is the problem memory or is it the code/library?

adwsystems:
Is the problem memory or is it the code/library?

Quite possibly in the code you didn't post.

Seems that the suggestion to try it on a processor with more memory might be a good start.

gfvalvo:
Quite possibly in the code you didn't post.

By 'code' I mean SDFat library. If it hangs right where it is shown in the OP, how can it be elsewhere?

gfvalvo:
Seems that the suggestion to try it on a processor with more memory might be a good start.

That assumes I have another board/processor readily available. I only had two MEGAs. One is in use and the other is dead and recycled. I will either have to wait for the system to become available so I can tear it apart, or wait for another to arrive.

The code as posted doesn’t compile. Therefore it is incomplete. Either post the full code or (preferably) a small MCVE that compiles and demonstrates the problem.

gfvalvo:
The code as posted doesn’t compile. Therefore it is incomplete. Either post the full code or (preferably) a small MCVE that compiles and demonstrates the problem.

I know some of you this this gambit. So here’s the full code. Good luck getting it to work. It does compile, obviously as the debug print statements are in the OP.

P.S. I will not be replying to any comments other than those directly related to the issue at hand.

SDFAT_Tester_post.ino (21.4 KB)

Only those serial prints that are for debugging are not using the F macro. All other serial print statements of constant value use the F macro.

Are you implying that because they are for debugging they don't consume RAM? How is the compiler supposed to know?

I know you think you've narrowed it down for us, but last I checked you don't know what the problem is. If you don't know what is causing this then how can you know what parts someone would need to see to find it? Sometimes the mistake is in what seems like a completely unrelated part of the code. You could have a buffer overrun in setup and that could be causing this weird behavior later. There are lots of things that don't seem to affect code the way you would think they would.

Delta_G:
Are you implying that because they are for debugging they don't consume RAM? How is the compiler supposed to know?

No I'm not implying that. I'm saying they will be removed and I'm not concerned about temporarily wasting 9 bytes. The problem occurred in the same place before the serial print statements, so their consumption is not the direct cause. Could they be making it worse? Yes, but no one has answered the question about memory.

Delta_G:
I know you think you've narrowed it down for us, but last I checked you don't know what the problem is. If you don't know what is causing this then how can you know what parts someone would need to see to find it? Sometimes the mistake is in what seems like a completely unrelated part of the code. You could have a buffer overrun in setup and that could be causing this weird behavior later. There are lots of things that don't seem to affect code the way you would think they would.

The complete code is posted. Post your findings.

adwsystems:
No I'm not implying that. I'm saying they will be removed and I'm not concerned about temporarily wasting 9 bytes. The problem occurred in the same place before the serial print statements, so their consumption is not the direct cause. Could they be making it worse? Yes, but no one has answered the question about memory.

Ok. Well you seem to have a good handle on it then. You must know what is causing the issue. I'm pretty sure you're running out of RAM but you seem to think you know that isn't the cause. So I'll leave you to it. Good luck.

But I have to ask, if you know what is wrong then why are you asking here?

Delta_G:
Ok. Well you seem to have a good handle on it then. You must know what is causing the issue. I'm pretty sure you're running out of RAM but you seem to think you know that isn't the cause. So I'll leave you to it. Good luck.

But I have to ask, if you know what is wrong then why are you asking here?

I don't know. Y'all want to go look through the rest of the program, great, have at it. The rest of the program runs just fine. Maybe there is an overrun error you will see that I missed. Why at THIS point in the program and ONLY this point in the program? Why does it run out of RAM here? Why at this specific point?

If it does the same thing on the MEGA, then what?

Why at THIS point in the program and ONLY this point in the program?

Buffer overruns and memory management issues lead to what is called undefined behavior. There is no “why at this point”. That’s just the way it worked out. Sometimes it has to do with how the compiler arranged things in memory. Sometimes other little things. Sometimes it gets so weird that you add a serial print in some completely unrelated part of the code and it changes the behavior. Or maybe even just changing what is printed ends up causing the program to start working or stop working.

With undefined behavior looking for the “why” is a fools errand. There’s no use chasing that down. You know what the issue is. Get to work fixing it. Start reducing memory usage and see if it helps.

Got the MEGA.

Compiler output:

Sketch uses 20904 bytes (8%) of program storage space. Maximum is 253952 bytes.
Global variables use 1722 bytes (21%) of dynamic memory, leaving 6470 bytes for local variables. Maximum is 8192 bytes.

Program still crashes at the same point.

This was written before your reply #15. Not sure if all of it still applies.

If you have memory issues, use the F macro regardless if it's a for debugging or real data. Not using it because you want to quickly check will make matters worse.

For a proper analysis [edit]of memory issues that you suspect are caused by low memory[/edit], you will have to analyse the generated assembly listing. See http://forum.arduino.cc/index.php?topic=406953.msg2801157#msg2801157 for the command. Redirect the output to a new file.

The elf file can be found in the build directory. I do not have all your libraries so can't compile your code.

Although I'm not an expert in that area, you can attach the resulting output file as well as the original source and I will have a look at it when I have time; no promises though.

The map file might be useful as well, but I've never looked at one.

Remember that from the 316 bytes, you take 20 for your time array when run_test() is called. All function calls inside run_test() will probably consume some memory as well.

OK, cherry on the cake (as it still crashes on a Mega). Check your array boundaries.

The below line can in theory result in a buffer overflow.

sprintf(line_text, "%-5s %2d.%1dV %01d.%01dA-Hr", temp_text, int((voltage_sel * 12) / 10), int((voltage_sel * 12) % 10), int(current_sel / 10), int(current_sel % 10));

I did not check if you limit any of your variables to a number of digits. In the below I made line_text slightly bigger to make sure that my code would not crash.

void setup()
{

  char line_text[30];
  char temp_text[21] = "COMPLETE";
  byte voltage_sel = 0xFF;
  byte current_sel = 0xFF;

  Serial.begin(57600);
  sprintf(line_text, "%-5s %2d.%1dV %01d.%01dA-Hr", temp_text, int((voltage_sel * 12) / 10), int((voltage_sel * 12) % 10), int(current_sel / 10), int(current_sel % 10));

  Serial.println(strlen(line_text));
}

void loop()
{
}

The strlen is 24 bytes. Your line_text is only 21 bytes long, needs to be 25.

sterretje:

void setup()

{

char line_text[30];
 char temp_text[21] = "COMPLETE";
 byte voltage_sel = 0xFF;
 byte current_sel = 0xFF;

Serial.begin(57600);
 sprintf(line_text, "%-5s %2d.%1dV %01d.%01dA-Hr", temp_text, int((voltage_sel * 12) / 10), int((voltage_sel * 12) % 10), int(current_sel / 10), int(current_sel % 10));

Serial.println(strlen(line_text));
}

void loop()
{
}



The strlen is 24 bytes. Your line_text is only 21 bytes long, needs to be 25.

Thanks for the catch. I will fix that but am not sure that is the problem (only problem?) as the program does not make it that far through the routine before it crashes.

You already have a globale line_text variable. Any reason for this?

void clear_display()
 {
  char line_text[21];  <<---