Debugging infrequent crash/heisenbug

[ps1066]

(I'm not sure this belongs in "Programming Questions", but it doesn't seem compatible with the description of "Installation and Troubleshooting". The description of the project is only to give context. I'm hoping to get some general debugging advice, not help with the project.)

I'm using an Arduino Wifi Rev 2, with the arduino IDE 1.8.19 (on an M1 MacBook Pro). I wrote a project using the IRRemote library to serve as a translator/repeater for IR controls for AV equipment. It works for a while, but eventually crashes (or stops functioning, anyway) in 1 of 2 ways. I use "crash" to mean "stops working": I don't actually know if execution has halted, or if the project is stuck in a loop.

1. I can immediately make it "crash" by sending a command from a particular IR remote. But when I put in serial prints to debug, it no longer crashes when I use that same remote. Since I don't need to use that particular remote in the actual application, in practice I could live it, but it would be really nice to understand the underlying problem since it might be related to 2).

2. In the real-world application, I have to leave it running without a serial connection. After some random time from hours to days, it stops working. There don't seem to be any particular external circumstances that correlate with it crashing.

Any advice on how to go about debugging in both these cases?

Is there any way to learn about the state of an arduino/project after a project has stopped functioning?

Is there some configuration change I can make when there is a serial printing to make it more likely to catch this bug that so far only seems to occur when there is no serial print? I've tried 9600 and 115200 baud, to no effect.

Many thanks,
Peter

[oldcurmudgeon]

The fact that adding serial debug "fixes" it suggests a couple of possibilities. A bad pointer or array reference which would incorrectly modify a different (less critical) area of memory after the addition. Another possibility is a timing problem that changes after adding the serial code.

Without the code it is difficult to tell.

[jremington]

Post the code, using code tags, and maybe someone can spot a common mistake.

Like using String objects on AVR based Arduinos, which causes random crashes.

[StefanL38]

posting your code will help a lot.
Another idea is if you have some spare-IO-pins to connect LEDs and switch the leds on/off in to specific combinations that are only set in one place inside your code

or by varying the line of code where you switch a single led on/off
This could help narrowing down where the problem occurs.

To check if memory corrupting is the reason add some variables that are really used so they really get compiled (and not optimised away by the compiler)
Adding the variable changes the memory-layout and if memory-corruption occurs it will occur in a different place making the code crashing not or different.

mostly memory corruption occurs if the write to arrays outside of the defined boundaries the array has.

best regards Stefan

[gilshultz]

Are you working with strings, if so that has been known to lead to memory leaks. I vaguely remember some posts here. It could also be a hardware problem. Can you post an annotated schematic as you have wired it including all grounds, power, power source(s).

[ps1066]

Thank you, All, very much for your suggestions.

The delay in my follow up was that, after I cleaned up my code to make it presentable for posting, the sketch ran for weeks without a problem before it finally crashed. After I hit the reset button, it only took a few days to crash again. After that it would only run for a day or so without crashing.

The only change to the code other than removing comments was removing a single Serial.println("Match"); that was only reached when the received IR code matched a target. I don't know the consequences of calling that when there is no serial line plugged in. I can't imagine there's any state that would persist after a reset (?), so perhaps the change in crash rate was spurious.

I also can't activate the original bug, where it crashed when one particular (rarely used) remote control was used. I think that may have actually gone away when I changed from IRRemote 3.7 to 3.8 a couple months back, but I conflated it with the other kind of crash. (The 3.8 release notes don't point to anything obviously relevant for my case.)

Today, I pulled the board out from behind the TV to draw the schematic. One possible issue is the Gikfun 8443 IR diode I had in series with the 510 Ohm resistor. The only purpose of this part was to see the signal as emitted by the TSAL 6200 with a scope probe. But maybe a stray light source could sag Vs enough to cause trouble?

I have removed the Gikfun part and the resistor. I'll report back on any stability improvement.

Note I am doing this on a breadboard, with ~5-10cm wires running direct into the Arduino I/O and voltage connector sockets.

Sketch:

#include <IRremote.h>

int RECV_PIN = 11;
int SEND_PIN = 3;

IRrecv irrecv(RECV_PIN);

IRsend irsend(SEND_PIN);


IRData vol_up1;

IRData vol_up2;


void setup()
{
   Serial.begin(115200);

  irrecv.enableIRIn(); // Start the receiver
  irsend.enableIROut(38); // Start the sender

  
  vol_up1.protocol=0x7;
  vol_up1.address=0x4;
  vol_up1.command=0x2;
  vol_up1.extra=0x0;
  vol_up1.numberOfBits=0x20;
  vol_up1.flags=0x0;
  vol_up1.decodedRawData=0xFD02FB04;

  vol_up2.protocol=0x8;
  vol_up2.address=0x4;
  vol_up2.command=0x2;
  vol_up2.extra=0x0;
  vol_up2.numberOfBits=0x0;
  vol_up2.flags=0x1;
  vol_up2.decodedRawData=0x0;
  
}

void loop() {
   
  if (irrecv.decode()) {

   if (irrecv.decodedIRData==vol_up1) {
      delay(100);
      sendSharpAudioIn();
    } 

   delay(100);   
   irrecv.resume(); // Receive the next value
  } 
}
void sendSharpAudioIn() {
  // ad hoc from capture of signal using decodeDistance()
    irsend.mark(3400);
    irsend.space(1650);
    irsend.sendPulseDistanceWidthData(450, 1250, 450, 400, 0x758F5AAA, 32, false, false);
    irsend.sendPulseDistanceWidthData(450, 1250, 450, 400, 0x3136, 16, false, true);
}
//

which uses my implementation of IRData::operator==

    bool operator==(const IRData& other) const {
         return protocol == other.protocol &&
                address == other.address &&
                command == other.command &&
                extra == other.extra &&
                numberOfBits == other.numberOfBits &&
                flags == other.flags &&
                decodedRawData == other.decodedRawData ;
    }

which is the only change I've made to IRRemote 3.8.

Thanks again,
Peter

[jremington]

Is the posted schematic correct?

An Arduino pin cannot usefully drive a high power IR diode, especially without a current limiting resistor.