Serial not resetting on power loss

Hi all,

I'm attempting to write a fairly robust system for an eventual exhibition piece and am trying to write in some fail safes in the case of catastrophic manhandling.

I'm interfacing with the serial port through a C program which utilises the "libserialport" library which is working fine other than I'm struggling with handling the event of the arduino being yanked out and plugged back in. Ideally, I'd like to have this automated to avoid a technician having to restart the computer running the exhibition software. I'm early on in development, so there are quite a few unnecessary stages which I'll omit once I'm past testing.

So far, it works fairly efficiently and handles the C program being restarted and power remaining to the arduino, but I'm getting unexpected bytes through the serial when the arduino has been unplugged and plugged back in.

On the script start I initialise the connection between the software and
arduino using a handshake byte: 'A'.

Once it's mirrored by the software,
the system moves into a ready_to_send state, emitting 'B', which when
mirrored by the software triggers the writing of a series of values, followed
by the system moving into the third state.

The third state waits for confirmation of receipt 'C'. On success, the system
reverts back to the ready_to_send state.

void loop() {
  reset = false;
  startSerial();
  state = 0;
  while (!reset) {

    switch (state) {
      case 0:
        // Establish connection;
        // Emitting: 'A' || 65
        state = establish_connection();
        // returns 1 if send cmd 'A' || 65 is received
        // returns 3 if received command is 'E' || 69
        break;
        
      case 1:
        // Waiting for send command
        // Emitting: 'B' || 66
        state = checkToSend();
        // returns 2 if send cmd 'B' || 66 is received
        // returns 3 if received command is 'E' || 69
        break;
        
      case 2:
        // Waiting for confirmation of receipt
        // --- Will program in some timeout to return
        // ---- back to connection state
        // Emitting: 'C' || 67
        state = checkReceived();
        // returns 1 if send cmd 'C' || 67 is received
        // returns 3 if received command is 'E' || 69
        break;
        
      case 3:
        reset = true;
        Serial.end();
        delay(1000);
        break;
    }
  }
}
int establish_connection() {
  // Sends out #65  waits for #65
  Serial.write('A');
  sendVal++;
  delay(500);
  toggleLight();
  for (i = 0; i < Serial.available(); i++) {
    response = Serial.read();
    if (response == 65 || response == 'A') {
      return 1;
    }
    if(checkForError(response)) {
      return 3;
    }
  }
  return 0;
}

When the arduino is unplugged and plugged back in I thought the arduino
would reboot and return back to the first state and send 'A' but all I'm receiving
is '\x 1 \x 1f " or which equates to "Start of Heading" and "Unit Header".

It seems like no matter how I restart the serial on the software side of things,
it remains stuck sending these bytes. The only way I've found to get it working
again is to start the arduino Serial Monitor. It's definitely in the first state as I've
set light patterns to identify which state it's in. Is there something the Serial
Monitor does other than restarting the port which would result in this?

I've attached both files for your browsing.

Cheers in advance!

int serialTest_parse_serial(char *byte_buff, int byte_num) {
	for (int i = 0; i < byte_num; i++) {
		switch (byte_buff[i]) {
		case READY_TO_CONNECT: {
			printf("Ready_to_connect cmd received");
			cmd2bSent[0] = 'A';
			if (serialTest_sendMsg(cmd2bSent, 1, 0)) {
				// Something went wrong
				printf("Something went wrong... attempting to write to serial...");
			}
			previousReceivedCMD = READY_TO_CONNECT;
			break;
		}

		case READY_TO_SEND:
			//printf("Ready_to_send cmd received");
			previousReceivedCMD = READY_TO_SEND;
			cmd2bSent[0] = 'B';
			serialTest_sendMsg(cmd2bSent, 1, 0);
			break;

		case CHECK_RECEIVED:
			previousReceivedCMD = CHECK_RECEIVED;
			//printf("check_received cmd received\n");
			cmd2bSent[0] = 'C';
			serialTest_sendMsg(cmd2bSent, 1, 0);
			break;

		case MSG_START:
			//printf("MSG_Start cmd received\n");
			previousReceivedCMD = MSG_START;
			receivingData = 1;
			break;

		case MSG_END:
			previousReceivedCMD = MSG_END;
			receivingData = 0;
			//for (int i = 0; i < 13; i++) {
			//	printf("%ld, ", receiveBuffer[i]);
			//}
			//printf("\n");
			//printf("\nLast Receive Count: %ld\n", dataCount);
			dataCount = 0;
			break;
		case CARRIAGE_R:
			//printf("CARRIAGE_R Cmd Received\n");
			// Restart Serial
			return -1;
		}

		if (receivingData == 1 && byte_buff[i] != MSG_START) {
			receiveBuffer[dataCount] = byte_buff[i];
			dataCount++;
		}

		if (byte_buff[i] == 1 ) {
			printf("Other received!");
			printf("%ld", byte_buff[i]);
			cmd2bSent[0] = 'E';
			serialTest_sendMsg(cmd2bSent, 1, 1);
			return -1;
		}

		return 1;
	}
}

FinalBSMM_debug.ino (4.32 KB)

serialTest.c (5.01 KB)

IF you know the bytes are "unexpected" why not toss them and continue?

What is the electrical protocol between the PC and the Arduino? RS-232, RS-485, TTL, etc?

Does your Arduino program initially set all pin to input, wait a bit and then set the output pins to output?

Are there any devices between the PC serial port and the Arduino?

Paul

If you have a PC program connected to your Arduino through a serial connection and the Arduino is unplugged and replugged (or otherwise caused to reset) it will probably appear on the PC as a different Serial port.

A possible solution is for your PC program to send a regular message to the Arduino and expect an immediate response. If there is no response then it could close the serial port and scan all the serial ports to see if it can find the Arduino on another one.

...R

Paul_KD7HB:
IF you know the bytes are "unexpected" why not toss them and continue?

What is the electrical protocol between the PC and the Arduino? RS-232, RS-485, TTL, etc?

Does your Arduino program initially set all pin to input, wait a bit and then set the output pins to output?

Are there any devices between the PC serial port and the Arduino?

Paul

Hi Paul, thanks for the response!

No devices in between, just a straight up USB connection. Would you advise on one of these as a more robust option?

The initial setup sets the PWM pins: 2 - 10 to output mode and then 22, 24... 48 of the digital pins to input, so no pin state switching as you noted. At the moment I've not added any of the final functionality so I can focus on stability.

The issue is more that the data being received on either end is what has been sent. i.e. on the pc side, I'm expecting the device to be sending one of 3 command bytes, 'A', 'B', 'C', to notify of its status and ensure that data is only sent when necessary to avoid locking up the serial buffer. On the other side, it's apparent that it's not receiving the right data as it's not changing state and responding by sending data. This is made apparent by the rx light flashing in time with commands being sent from the software and there being no change in state.

Robin2:
If you have a PC program connected to your Arduino through a serial connection and the Arduino is unplugged and replugged (or otherwise caused to reset) it will probably appear on the PC as a different Serial port.

A possible solution is for your PC program to send a regular message to the Arduino and expect an immediate response. If there is no response then it could close the serial port and scan all the serial ports to see if it can find the Arduino on another one.

...R

Hi Robin,

Thanks for the response!
Yes, that's actually something I had forgotten to check. However, the device is still sending data over the COM6 port it's assigned prior to being replugged. Also, the rx light shows that the software is communicating with the arduino. I find that the arduino rarely shifts from COM6, as COM4 AND COM5 are always utilised.

I wonder if it's a case of resetting the COM ports through a script - I've just found this:
https://support.microsoft.com/en-gb/help/819036/overview-of-the-comdisable-tool
However this seems pretty drastic seeing as the serial monitor manages to communicate some command to the arduino which triggers some change in the serial port.

cheers!

One thing I have noticed is that in the serial monitor, it shows for the first two printouts the \x 1 and \x 1f bytes, pauses for less than a second and then starts receiving the expected "AAAAAAAAAAAA".

Some reset is happening which I can't replicate in the libserialport library.

stuartmmellor:
However, the device is still sending data over the COM6 port it's assigned prior to being replugged. Also, the rx light shows that the software is communicating with the arduino. I find that the arduino rarely shifts from COM6, as COM4 AND COM5 are always utilised.

That should make life a lot simpler.

Indeed, if that is true I don't understand why there is a problem, or exactly what the problem is.

Perhaps it is the case that things have been working normally and have got to the stage where your PC program is expecting K and the Arduino (which has just reset) sends A but your program can't cope with that?

If that is the sort of problem you have maybe it would help if the Arduino sends a special standard message when it starts (or resets) so that your PC program can recognise that and re-configure itself. By the way I am not suggesting that recovery is necessarily simple. It may be much better use of your time and resources to make the problem less likely.

...R

stuartmmellor:
Hi Paul, thanks for the response!

No devices in between, just a straight up USB connection. Would you advise on one of these as a more robust option?

The initial setup sets the PWM pins: 2 - 10 to output mode and then 22, 24... 48 of the digital pins to input, so no pin state switching as you noted. At the moment I've not added any of the final functionality so I can focus on stability.

The issue is more that the data being received on either end is what has been sent. i.e. on the pc side, I'm expecting the device to be sending one of 3 command bytes, 'A', 'B', 'C', to notify of its status and ensure that data is only sent when necessary to avoid locking up the serial buffer. On the other side, it's apparent that it's not receiving the right data as it's not changing state and responding by sending data. This is made apparent by the rx light flashing in time with commands being sent from the software and there being no change in state.

If you were using a standard RS-232 serial port, your system would work. By using a USB for "serial" communications, you are not seeing all the negotiation communication between the PC and the Arduino to first make the USB work and second to make the serial work over usb. Review the USB standards for all that must transpire before it becomes usable by the two devices.

Paul

The good news is that I've managed to reproduce the issue of 01 1F using my own C/C++ code; it will not always be 01 1F though on my system.

Note that I'm not a very experienced C/C++ programmer under Windows.

// MySerialMonitor.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <Windows.h>

void print_error(const char * context);
HANDLE open_serial_port(const char * device, UINT32 baud_rate);
SSIZE_T read_port(HANDLE port, UINT8 *buffer, size_t size);
int write_port(HANDLE port, UINT8 * buffer, size_t size);

int main()
{
	const char * device = "\\\\.\\COM5";
	UINT32 baud_rate = 19200;

	HANDLE port = INVALID_HANDLE_VALUE;
	int retries = 0;
	while (true)
	{
		if (port == INVALID_HANDLE_VALUE)
		{
			printf("opening %s\r\n", device);
			port = open_serial_port(device, baud_rate);
			if (port == INVALID_HANDLE_VALUE)
			{
				retries++;
				if (retries == 10)
				{
					return -1;
				}
				Sleep(2000);
				continue;
			}
			retries = 0;
			printf("%s open\r\n", device);
		}

		UINT8 buffer[256];
		memset(buffer, 0xFF, sizeof(buffer));
		int rc = read_port(port, buffer, 1);
		if (rc == -1)
		{
			CloseHandle(port);
			port = INVALID_HANDLE_VALUE;
			continue;

		}

		//printf("[%d]0x%02X \r\n", rc, buffer[0]);
		if (rc != 0)
		{
			printf("[%d]0x%02X \r\n", rc, buffer[0]);
			if (buffer[0] == 'A' || buffer[0] == 'B' || buffer[0] == 'C')
			{
				rc = write_port(port, buffer, 1);
				if (rc == -1)
				{
					CloseHandle(port);
					port = INVALID_HANDLE_VALUE;
				}
			}
		}
	}

	CloseHandle(port);
	
	return 0;
}



void print_error(const char * context)
{
	DWORD error_code = GetLastError();
	char buffer[256];
	DWORD size = FormatMessageA(
		FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_MAX_WIDTH_MASK,
		NULL, error_code, MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US),
		buffer, sizeof(buffer), NULL);
	if (size == 0) { buffer[0] = 0; }
	fprintf(stderr, "%s: %s\n", context, buffer);
}

// Opens the specified serial port, configures its timeouts, and sets its
// baud rate.  Returns a handle on success, or INVALID_HANDLE_VALUE on failure.
HANDLE open_serial_port(const char * device, UINT32 baud_rate)
{
	HANDLE port = CreateFileA(device, GENERIC_READ | GENERIC_WRITE, 0, NULL,
		OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
	if (port == INVALID_HANDLE_VALUE)
	{
		print_error(device);
		return INVALID_HANDLE_VALUE;
	}

	// Flush away any bytes previously read or written.
	BOOL success = FlushFileBuffers(port);
	if (!success)
	{
		print_error("Failed to flush serial port");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	// Configure read and write operations to time out after 100 ms.
	COMMTIMEOUTS timeouts = { 0 };
	timeouts.ReadIntervalTimeout = 0;
	timeouts.ReadTotalTimeoutConstant = 100;
	timeouts.ReadTotalTimeoutMultiplier = 0;
	timeouts.WriteTotalTimeoutConstant = 100;
	timeouts.WriteTotalTimeoutMultiplier = 0;

	success = SetCommTimeouts(port, &timeouts);
	if (!success)
	{
		print_error("Failed to set serial timeouts");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	DCB state;
	state.DCBlength = sizeof(DCB);
	success = GetCommState(port, &state);
	if (!success)
	{
		print_error("Failed to get serial settings");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	state.BaudRate = baud_rate;

	success = SetCommState(port, &state);
	if (!success)
	{
		print_error("Failed to set serial settings");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	return port;
}

// Returns the number of bytes successfully read into the buffer, or -1 if
// there was an error reading.
SSIZE_T read_port(HANDLE port, UINT8 *buffer, size_t size)
{
	DWORD received;
	BOOL success = ReadFile(port, buffer, size, &received, NULL);
	if (!success)
	{
		print_error("Failed to read from port");
		return -1;
	}
	return received;
}

// Writes bytes to the serial port, returning 0 on success and -1 on failure.
int write_port(HANDLE port, UINT8 * buffer, size_t size)
{
	DWORD written;
	BOOL success = WriteFile(port, buffer, size, &written, NULL);
	if (!success)
	{
		print_error("Failed to write to port");
		return -1;
	}
	if (written != size)
	{
		print_error("Failed to write all bytes to port");
		return -1;
	}
	return 0;
}

When the code receives an 'A' or 'B' or 'C', it will echo the character to the Arduino.

The bad news is that you have to solve it yourself in the library that you use; I did manage to solve it in the above code by setting the ByteSize (number of databits). I added some additional stuff; basically all relevant elements of the DCB need to be initialised.

HANDLE open_serial_port(const char * device, UINT32 baud_rate)
{
	HANDLE port = CreateFileA(device, GENERIC_READ | GENERIC_WRITE, 0, NULL,
		OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
	if (port == INVALID_HANDLE_VALUE)
	{
		print_error(device);
		return INVALID_HANDLE_VALUE;
	}

	// Flush away any bytes previously read or written.
	BOOL success = FlushFileBuffers(port);
	if (!success)
	{
		print_error("Failed to flush serial port");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	// Configure read and write operations to time out after 100 ms.
	COMMTIMEOUTS timeouts = { 0 };
	timeouts.ReadIntervalTimeout = 0;
	timeouts.ReadTotalTimeoutConstant = 100;
	timeouts.ReadTotalTimeoutMultiplier = 0;
	timeouts.WriteTotalTimeoutConstant = 100;
	timeouts.WriteTotalTimeoutMultiplier = 0;

	success = SetCommTimeouts(port, &timeouts);
	if (!success)
	{
		print_error("Failed to set serial timeouts");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	DCB state;
	state.DCBlength = sizeof(DCB);
	success = GetCommState(port, &state);
	if (!success)
	{
		print_error("Failed to get serial settings");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	state.BaudRate = baud_rate;

	/*************
	initialise DCB to 8N1
	*************/
	state.ByteSize = 8;
	state.Parity = NOPARITY;
	state.StopBits = ONESTOPBIT;
	
	/*************
	prevent Arduino's reset when port opens
	*************/
	state.fDtrControl = false;
	state.fRtsControl = false;

	success = SetCommState(port, &state);
	if (!success)
	{
		print_error("Failed to set serial settings");
		CloseHandle(port);
		return INVALID_HANDLE_VALUE;
	}

	return port;
}

You will have to try to find this part in the library that you use and check; or drop the library and base your code on above.

Below output after the fix; failed to read from port is the point where I disconnected the Arduino. Tested on both a Sparkfun RedBoard (FTDI chip) and original Uno (16U2).

opening \\.\COM5
\\.\COM5 open
[1]0x41
[1]0x42
[1]0x2B
[1]0x31
[1]0x32
[1]0x33
[1]0x34
[1]0x35
[1]0x36
[1]0x37
[1]0x38
[1]0x39
[1]0x3A
[1]0x3B
[1]0x3C
[1]0x3D
[1]0x2D
[1]0x43
[1]0x42
[1]0x2B
[1]0x31
[1]0x32
[1]0x33
[1]0x34
[1]0x35
[1]0x36
[1]0x37
[1]0x38
[1]0x39
[1]0x3A
[1]0x3B
[1]0x3C
[1]0x3D
[1]0x2D
[1]0x43
Failed to read from port: The I/O operation has been aborted because of either a thread exit or an application request. 
opening \\.\COM5
\\.\COM5: The system cannot find the file specified.
opening \\.\COM5
\\.\COM5: The system cannot find the file specified.
opening \\.\COM5
\\.\COM5 open
[1]0x41
[1]0x42
[1]0x2B
[1]0x31
[1]0x32
[1]0x33
[1]0x34
[1]0x35
[1]0x36
[1]0x37
[1]0x38
[1]0x39
[1]0x3A
[1]0x3B
[1]0x3C
[1]0x3D
[1]0x2D
[1]0x43
Failed to read from port: The I/O operation has been aborted because of either a thread exit or an application request.

PS 1
Code compiled with VS2017 community edition.
PS 2
Code based on Pololu - 15.6. Example serial code for Windows in C
PS 3
I slowed down the Arduino code with an additional 1 second delay

  while (!reset)
  {
    delay(1000);

Hi Sterretje thanks for the response!!

That's excellent, I've slightly overlooked the databits on the C software side of things.

I think that the library I'm using - libserialport - sets this by default, but I'll check.
It might be that the library is doing some asynchronous instantiation and isn't successfully setting the byte size.

If not, thanks for your implementation!