Arduino OPTA crashes after a few months

Official Hardware Opta
1

Hi all,

I’m using an Arduino OPTA PLC programmed through the Arduino PLC IDE (version 1.0.8.0). It’s connected to a relay/IO extension module, and I mainly use the Arduino PLC runtime with its built-in MODBUS TCP slave interface to communicate with the device I/Os.

The host system polls the PLC inputs once per second via MODBUS over TCP.

However, every few months, the communication to the PLC crashes. When this happens:

  • The PLC becomes unreachable — I can no longer ping it or establish any MODBUS connection. (I tried from another system on the network)
  • All relays turn off (normally, at least one relay should always be active), suggesting the PLC has entered some internal fault or reset state.

Power-cycling the PLC restores normal operation immediately. MODBUS communication then works fine again (both from the host and from manual tests on my PC), and the system continues running normally for several months, until the issue reoccurs.

Has anyone experienced similar behavior?
Could this be a firmware or runtime reliability issue?
Any suggestions on how to debug or permanently fix this would be nice.
Can I implement some type of watchdog to reset upon a crash?

Thanks in advance!

2

I have no experience with the Opta, but the problem sounds like a slow memory leak that eventually consumes all of RAM. Monitor memory usage, free memory or both over time to check for that.

3

This code will override the default mbed_die() function and reset the system if a hard fault occurs.

MBED_NORETURN void mbed_die () {
  gpio_t led_red, led_grn, led_blu;
         
  gpio_init_out(&led_red, LEDR);
  gpio_init_out(&led_grn, LEDG);
  gpio_init_out(&led_blu, LEDB);

  gpio_write(&led_red, 1);
  gpio_write(&led_grn, 1);
  gpio_write(&led_blu, 1);

  for (int i = 0; i < 2; ++i) {
    for (int j = 0; j < 4; ++j) {
      gpio_write(&led_red, 0);
      wait_us(150000);
      gpio_write(&led_red, 1);
      wait_us(150000);
    }
    for (int j = 0; j < 4; ++j) {
      gpio_write(&led_red, 0);
      wait_us(400000);
      gpio_write(&led_red, 1);
      wait_us(400000);
    }
  }
  NVIC_SystemReset();
}
4

Maybe some sort of overflow? How many seconds or milliseconds in the time between crashes?

1 Like
5

Thanks, So I can just drop the mbed_die() function inside the "sketch" where the setup() and loop() functions are defined? It compiled succesfully.

Is there a way I can raise a hang manually, to check if the handler works?

image
image1237×506 27.1 KB

6

Months!! previous crash was in july.

7

Yes, that's fine.

If you call this code, it'll trigger a divide by zero exception.

    SCB->CCR |= SCB_CCR_DIV_0_TRP_Msk;
    int x = 1;
    int y = 0;
    int s = x / y;
    printf("s=%d",s);
    return;
8

Then more likely a memory leak.

9

Thank you.

It's currently still in its hang state (didn't yet power cycle it). Can i get its current trace through the USB to get some more debug information? Is there any logging done that may pinpoint the crash?

I'll implement the code and see if it happens again. If its a memory leak, its part of the runtime environment, right?

10

Not on a GIGA R1 (the board I use), and probably not on the OPTA (unless the PLC code does something clever).

Could be your code, Arduino's or mbed-os'.

You could use something like this in your dev environment to monitor free heap memory.

mbed_stats_heap_t heap_stats;
...
mbed_stats_heap_get(&heap_stats);
Serial.println(heap_stats.reserved_size - heap_stats.current_size);

If you do something like this in your dev setup() you can lock out a large chunk of your heap so that the allocation failure (if that's what it is) would occur much sooner.

char *lockout = new char[350000];

p.s. For completeness, I should mention that crash reporting data can be made available, but it's a lot of work. See section Crash reporting and auto-reboot

11

Thanks

I also included the WDT as follows. However, I'm still not capturing issues with the ethernet PHY or TCPIP stack. The OS may keep running but the networking may fail. Can I also monitor that?

#include <mbed.h>
mbed::Watchdog &watchdog = mbed::Watchdog::get_instance();


void setup()
{
    watchdog.start(1000);
	// Configure static IP address
	IPAddress ip(192, 168, 1, 105);
	IPAddress dns(8, 8, 8, 8);
	IPAddress gateway(192, 168, 1, 104);
	IPAddress subnet(255, 255, 255, 0);
	// If cable is not connected this will block the start of PLC with about 60s of timeout!
	Ethernet.begin(ip, dns, gateway, subnet);
}

void loop()
{
    delay(500);
    watchdog.kick();

}
MBED_NORETURN void mbed_die () {
  NVIC_SystemReset();
}
12

I'm unable to offer much in that area as I don't use ETH.

The Ethernet class has member functions linkStatus() and hardwareStatus() that may indicate when there's a problem. ArduinoCore-mbed/libraries/Ethernet/examples/LinkStatus/LinkStatus.ino at main · arduino/ArduinoCore-mbed · GitHub

You could monitor socket stats but that would need a tweak to your mbed config and a rebuild of the mbed core lib.a

One option that may be available to you is a scheduled auto-reboot once a month. Crude workaround but might be acceptable if you can't get to the bottom of this.

p.s. this thread from last year may be of interest, if only to touch base with other OPTA users that have had similar problems Arduino Opta Reset LED Flashing Red 4 times on/off | Network issues