Extend logging from IDE

Team,

Built a remote 4G sensor and its been installed and working for approx 2-3 months then went off line and never came back online until I powered off and on again. Now its working as expected.

Q. As I have it back in my test lab what is the best way to figure out if this is a one off or not? How can I use the serial log to log for a extended period of time?

Hi @droidblast.

You can use a standalone serial terminal application that writes the received data to a file.

There are several nice free terminal programs available that provide this capability. I can recommend the open source PuTTY for this purpose:

https://www.chiark.greenend.org.uk/~sgtatham/putty/

Note that you will need to close the serial port in the standalone terminal before performing uploads in Arduino IDE so that the terminal's connection to the port doesn't block the IDE's ability to use it.

In my opinion Putty is rather complicated tool because it is a unisversal tool one for all. Not only for serial connections.

Putty offers a lot of options and is very suitable for remotely maintaining other computers. This versatility requires to learn how to use it. There is a certain effect that always happens. And there is only one way to overcome this effect.

I call this effect I call it the experts blindness for beginner's difficulties.

Compared to putty hTerm is very simple to use

The reason I choose to recommend PuTTY is because it is open source.

Thanks tried both and using putty as I am use to it but both work.

Q. Logging is building the log file and I can use PS to tail just to check...

Get-Content D:\Putty\batchprotect.log –Wait

I see some hieroglyphics in the logs yet the baud rate and 115200 8N1 is correct.

Are the logs entirely garbage, or is there only some garbage in addition to the expected content?

If you close the port in PuTTY and then open it in the Arduino IDE Serial Monitor, do you see the same type of content as you got from PuTTY?

Both show the same readable logs with maybe one or two random hieroglyphics that dont look to repeat at the same intervals. The Baud/8n1 is correct and I do see what I expect to see, with the exception of random odd hieroglyphics from time to time. I am using 4G not wireless to send mainly MQTT topics and subs both ways from and to the device.

The reason I asked how to long term serial log is to see what the device went out of contact.

Is it good practice to leave debug logging on once you have tested then deployed a device?

Thanks for the clarification. It might have various causes. For example:

  • It is the actual data
  • A bug in your code
  • A hardware problem

The forum helpers might be able to identify the specific cause and suggest a solution, but they would likely need to have your sketch code as well as details about your hardware.

There is overhead from the logging, both in terms of memory usage and clock cycles. So if we aren't using it at all and the overhead is of concern (e.g., fitting the program into the available memory is tight, program execution speed is important, low power application) then we would typically disable the logging code when it is not needed.


Here is a system you can use to quickly enable and disable the debug output:

Add this code near the top of your sketch:

#define DEBUG true  // Set to true for debug output, false for no debug output.
#define DEBUG_SERIAL \
  if (DEBUG) Serial

Then use code like this to initialize the serial communication:

DEBUG_SERIAL.begin(9600);

If you want your program to wait for Serial Monitor to be opened before running when using native USB boards (e.g., Leonardo), add this line:

#if DEBUG == true
  while (!Serial) {
    ;  // wait for serial port to connect. Needed for native USB port only
  }
#endif  // DEBUG == true

and code like this for debug output:

DEBUG_SERIAL.println("Some debug output");

When the DEBUG macro is set to false, the compiler will optimize the calls using DEBUG_SERIAL out of the code because it knows they will never run. This means the debug output code won't use up any memory and won't slow down the execution of the program.

In the rare case where your debug system needs to read serial input, you would use similar code:

#if DEBUG == true
  if (Serial.available()) {
    x = Serial.read();
  }
#endif  // DEBUG == true

This system can easily be extended to allow multiple levels of debug output, still with no overhead when it's disabled:

#define DEBUG_ERROR true
#define DEBUG_ERROR_SERIAL \
  if (DEBUG_ERROR) Serial

#define DEBUG_WARNING true
#define DEBUG_WARNING_SERIAL \
  if (DEBUG_WARNING) Serial

#define DEBUG_INFORMATION true
#define DEBUG_INFORMATION_SERIAL \
  if (DEBUG_INFORMATION) Serial

void setup() {
  Serial.begin(9600);
  while (!Serial) {
    ;  // wait for serial port to connect. Needed for native USB port only
  }
  DEBUG_ERROR_SERIAL.println("This is an error message");
  DEBUG_WARNING_SERIAL.println("This is a warning message");
  DEBUG_INFORMATION_SERIAL.print("The state of pin 5 is ");
  DEBUG_INFORMATION_SERIAL.println(digitalRead(5) ? "HIGH" : "LOW");
  Serial.println("This is standard program output");
}

void loop() {}

You can use the Boolean debug macros to switch on and off other parts of your code too.

Thanks for the feedback. I made some changes in haste and the redeployed the code with minor mods. One mod is wrong e.g., its reports the state of the device back to the main unit. This is actually wrong and toggling the current state. I need to revise this and then let it run for a month plus. The sketch is close to 2k of code and I think I will disable debugging post this last change.

You are welcome. I'm glad if I was able to be of assistance.

Regards,
Per

Quick update...

Still have the same issue September 17th to October 9th so thats 23 days of runtime.

Project is at a site 4hrs away and I have disabled the serial output as well so I will need to physically drove there to restart and start logging serially again but....

I run a fair few functions like this for example...

int TempCount = 0;

void Update_temp()                                                                   
{
  TempCount++;                                                                       
  if (TempCount >= 43200) {                                                          
    TempCount = 0;
    temp = modem.getTemperature();                                                   
    char tempString[8];                                                              
    dtostrf(temp, 1, 2, tempString);
    mqtt.publish(topicTemperature, tempString);                                      
  }
}

So am I on the right track here?

If TempCount is unsigned int (16-bit unsigned): An unsigned int can hold a maximum value of 65,535. With Update_temp() called once per second: The counter would overflow after 65,535 seconds.

Converting this to days:
65,535 seconds ÷ (60 × 60) = ~18.2 hours.
So, using unsigned int would extend the overflow time to about 18 hours, but this is still far less than 23 days, leading to communication loss well before the end of the period.

If TempCount is unsigned long (32-bit unsigned):
An unsigned long can hold values up to 4,294,967,295.
With Update_temp() called once per second: The counter would overflow after 4,294,967,295 seconds.

Using an unsigned long would give a much longer time before overflow, and the 23 days of operation is well within the capacity of this data type.

Unsigned long: Use for counters that need to handle long time periods (e.g., over several days or weeks).

Unsigned int: If you know that a counter will never exceed 65,535 (around 18 hours for 1-second increments), you could consider using unsigned int instead to save memory (if your microcontroller has limited RAM).

Change to ...

Using unsigned long allows these counters to handle much larger values (up to 4,294,967,295) without risk of overflow for an extended period (years, depending on increment frequency).

unsigned long TempCount = 0;
unsigned long TempCount2 = 0;
unsigned long OverTempCount = 0;
unsigned long DelaytempCount = 0;
unsigned long CheckBalanceCount = 0; 
unsigned long VoltagePSCount = 0; 
unsigned long VoltagePSCount2 = 0;
unsigned long SDCardCount = 0;
unsigned long SDCardCount2 = 0;
unsigned long CSGtempCount = 0; 
unsigned long Check_SMSCount = 0;

Variables are just flags that store HIGH or LOW or true/false states, consider using bool to reduce memory usage.

Boolean Values (PAlarm, LStatus, PoffStatus, Ponoff):

bool PAlarm = LOW;
bool LStatus = LOW;
bool PoffStatus = LOW;
bool Ponoff = LOW;

So am I heading in the right direction?

bool is used for simple true/false decisions.
unsigned int is used for medium-range positive numbers, up to 65,535 on most microcontrollers (or higher on 32-bit systems).
unsigned long is used when you need a larger range of positive numbers, up to 4.2 billion, such as for long-duration timers or large counters.

Don't you think that such a project is worth using a microcontroller with much more RAM than just 2 kB?

Or adding another micocontroller over serial interface and this second microcontroller has an SD-Card connected to save you gigabytes of logdata ?

Or even adding another microcontroller with an additional SIM-card which would enanble to switch on / off reset-signals from the distance?

What are the overall circumstances of this project?
Is this a battery-powered sensor high up in the mountains where you have to climb up on your own feet?

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.