Trouble with Serial.readBytes on Windows vs Linux

ReqPro1 · September 13, 2022, 10:19am

Hi all,

I've got this little .NET 6.0 program that sends a simple text string to the Arduino, that then processes this string and displays something on a TFT screen.

This all works when sending the string from a Windows computer - text is processed and shown correctly on the screen.

However, when this program is compiled and executed on the Linux machine (Ubuntu), the Arduino seems to simply crash and reboot.
I've done some logging code that saves some debug messages on a SD card, and I can tell that the code crashes exactly when the bytes are being read from Serial (no other log messages directly after that).

Note that this issue persists, regardless of the baudrate - I've tried everything between 9600 and 500000

Any insights?

Here's my arduino code for receiving data:

#include <arduino.h>
#include <SPI.h>             // f.k. for Arduino-1.5.2
#define USE_SDFAT
#include <SdFat.h>           // Use the SdFat library
#include <MCUFRIEND_kbv.h>
#include "config.h"
#include "RowData.h"

#if SPI_DRIVER_SELECT != 2
#error edit SdFatConfig.h .  READ THE SKETCH INSTRUCTIONS
#endif

SoftSpiDriver<12, 11, 13> softSpi; //Bit-Bang on the Shield pins SDFat.h v2
SdFat SD;
SdFile myFile;
#define SD_CS SdSpiConfig(10, DEDICATED_SPI, SD_SCK_MHZ(0), &softSpi)

MCUFRIEND_kbv tft;
const int bufferSize = 512; 

char buffer[bufferSize];
RowData* rowData = 0;
String* rowBuffer = 0;

void setup()
{
    if (!SD.begin(SD_CS)) SD.initErrorHalt();
    
    uint16_t ID;
    Serial.begin(baudRate);
    ID = tft.readID();
    log("Found TFT with ID:0x" + String(ID, HEX));
    if (ID == 0x0D3D3) ID = 0x9481;
    tft.begin(ID);

    tft.setRotation(displayRotation); // Default: 0 (portrait)
    tft.setTextWrap(false); // Default: True
    tft.setTextColor(fontColor); 
    tft.setTextSize(fontScale); // Default: 1
    tft.fillScreen(background);

    log("Setup complete.");
}

void loop()
{
    log("Waiting for input...");

	while (!Serial.available());
	int bytesRead = Serial.readBytes(buffer, bufferSize);
	buffer[bytesRead] = '\0';

    log("Received " + String(bytesRead, DEC) + " bytes");
	
    tft.fillScreen(background);
    .........
}

UKHeliBob · September 13, 2022, 10:27am

  int bytesRead = Serial.readBytes(buffer, bufferSize);
  buffer[bytesRead] = '\0';

bufferSize is 512 so bytesRead will be 512, but then you put a value into buffer[512] which does not exist

This is wrong whatever OS you are using

ReqPro1 · September 13, 2022, 10:30am

I'm not sure I know what you mean here...
I'm passing the maximum buffer size to Serial.readBytes so it won't cause a buffer overflow.
Serial.readBytes will read all available data up to 512 bytes, because the buffer can't hold anymore data than that. It then returns the size of the data read.
It is impossible to pass on the size of data to receive into Serial.readBytes as it is unknown at this point...
At least that's how I'm understanding it?
Also, bytesRead is never 512 when I manage to run it on the Windows platform - instead it matches perfectly the size of data sent through the serial.

UKHeliBob · September 13, 2022, 10:32am

It is not Serial.readBytes() that is causing the problem

What is the highest index number of the buffer array and which index number are you using when you do

buffer[bytesRead] = '\0';

Danois90 · September 13, 2022, 10:33am

You could be accessing memory which is out of bounds, bytesRead may be equal to bufferSize which is 512, but buffer[512] is not within the bounds (0..511):

int bytesRead = Serial.readBytes(buffer, bufferSize - 1);
buffer[bytesRead] = '\0';

Whandall · September 13, 2022, 10:34am

Seems the line ending differences between Linux and Windows cause your problem.

Linux uses only one char, windows two.

Maybe you search for the wrong delimiter.

ReqPro1 · September 13, 2022, 10:38am

I totally see your point here....
If bytesRead is 512 and I'm trying to set buffer[512] to \0 that will of course cause an index out of range, since the last slot in buffer is 511...

I will try to alter that right away - good catch

Danois90 · September 13, 2022, 10:40am

And also have in mind that a line ending in windows is "\r\n" whereas it is "\n" in Linux as suggested by @Whandall. When parsing the content received from .readBytes(), simply ignore '\r' and use '\n' as line ending.

ReqPro1 · September 13, 2022, 10:42am

This is also a good point, but I'm not processing line endings at all in my code, so these should be completely irrelevant

sterretje · September 13, 2022, 10:49am

What's the rest of your loop()?

Did you compile and upload the Arduino code on the Linux system or is your Arduino running the code that was loaded using a Windows system?

What is your mystery .NET program?

ReqPro1 · September 13, 2022, 11:10am

The mystery .NET program is irrelevant as I have just tried to boil it down to the bare minimums...

For this purpose, here's the Arduino code:

#include <arduino.h>
#include <SPI.h>             // f.k. for Arduino-1.5.2
#define USE_SDFAT
#include <SdFat.h>           // Use the SdFat library
#include <MCUFRIEND_kbv.h>
#include "config.h"

#if SPI_DRIVER_SELECT != 2
#error edit SdFatConfig.h .  READ THE SKETCH INSTRUCTIONS
#endif

SoftSpiDriver<12, 11, 13> softSpi; //Bit-Bang on the Shield pins SDFat.h v2
SdFat SD;
SdFile myFile;
#define SD_CS SdSpiConfig(10, DEDICATED_SPI, SD_SCK_MHZ(0), &softSpi)

MCUFRIEND_kbv tft;
const int bufferSize = 512; 

char buffer[bufferSize];

void setup()
{
    if (!SD.begin(SD_CS)) 
    {
        Serial.print("SD Error!");
        SD.initErrorHalt();
    }
    
    uint16_t ID;
    Serial.begin(baudRate);
    ID = tft.readID();
    log("Found TFT with ID:0x" + String(ID, HEX));
    if (ID == 0x0D3D3) ID = 0x9481;
    tft.begin(ID);

    tft.setRotation(displayRotation); // Default: 0 (portrait)
    tft.setTextWrap(false); // Default: True
    tft.setTextColor(fontColor); 
    tft.setTextSize(fontScale); // Default: 1
    tft.fillScreen(background);

    log("Setup complete.");
}

void loop()
{
    log("Waiting for input...");

    while (!Serial.available());

    int bytesRead = Serial.readBytes(buffer, bufferSize - 1);
    buffer[bytesRead] = '\0';	

    log("Received " + String(bytesRead, DEC) + " bytes");
	
    tft.fillScreen(background);
}

And the .NET code:

class Program
{
    private static Settings _settings = new();

    public static void Main(params string[] args)
    {
        IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .AddEnvironmentVariables()
        .Build();

        _settings = config.GetRequiredSection("Settings").Get<Settings>();
        if (_settings == null || string.IsNullOrEmpty(_settings.RomBaseDir))
        {
            Console.WriteLine("Invalid or missing appsettings.json file");
            Environment.Exit(0);
            return;
        }

        Test.Run(_settings);
    }
}

static class Test
{
	public static void Run(Settings settings)
	{
		foreach (var comport in settings.GraphicsDisplayComPorts)
		{
			try
			{
				using (var sp = new SerialPort())
				{
					sp.PortName = comport; // /dev/ttyACM0 for linux - COM8 for Windows
					sp.BaudRate = settings.BaudRate; // 9600
					sp.Open();
					sp.WriteLine("Test string");
					sp.Close();
				}
			}
			catch (Exception e)
			{
				Console.WriteLine("Failed to communicate with display on port " + comport + ":");
				Console.WriteLine(e);
			}
		}
	}
}

These produce the exact same results: it works when the .NET program is run from Windows, but crashes the Arduino when run from Linux.

ReqPro1 · September 13, 2022, 11:11am

Unfortunately, this didn't fix it

UKHeliBob · September 13, 2022, 11:18am

Please post your full revised sketch

ReqPro1 · September 13, 2022, 11:29am

Here's the Arduino sketch:

#include <arduino.h>
#include <SPI.h>             // f.k. for Arduino-1.5.2
#define USE_SDFAT
#include <SdFat.h>           // Use the SdFat library

#if SPI_DRIVER_SELECT != 2
#error edit SdFatConfig.h .  READ THE SKETCH INSTRUCTIONS
#endif

SoftSpiDriver<12, 11, 13> softSpi; //Bit-Bang on the Shield pins SDFat.h v2
SdFat SD;
SdFile myFile;
#define SD_CS SdSpiConfig(10, DEDICATED_SPI, SD_SCK_MHZ(0), &softSpi)

const uint32_t baudRate = 9600;
const int bufferSize = 512; 
char buffer[bufferSize];

void setup()
{
    if (!SD.begin(SD_CS)) 
    {
        Serial.print("SD Error!");
        SD.initErrorHalt();
    }
    
    Serial.begin(baudRate);
    log("Setup complete.");
}

void loop()
{
    log("Waiting for input...");

	while (!Serial.available());
	int bytesRead = Serial.readBytes(buffer, bufferSize - 1);
    buffer[bytesRead] = '\0';	

    log("Received " + String(bytesRead, DEC) + " bytes");
}

void log(String text)
{
    Serial.println(text);

    if (!myFile.open("log.txt", O_RDWR | O_CREAT | O_AT_END )) SD.errorHalt("opening log.txt for write failed");
    myFile.println(text);
    myFile.close();
}

And the C# code:

    class Program
    {
        private static Settings _settings = new();

        public static void Main(params string[] args)
        {
            IConfiguration config = new ConfigurationBuilder()
            .AddJsonFile("appsettings.json")
            .AddEnvironmentVariables()
            .Build();

            _settings = config.GetRequiredSection("Settings").Get<Settings>();
            if (_settings == null || string.IsNullOrEmpty(_settings.RomBaseDir))
            {
                Console.WriteLine("Invalid or missing appsettings.json file");
                Environment.Exit(0);
                return;
            }

            foreach (var comport in _settings.GraphicsDisplayComPorts)
            {
	            try
	            {
		            using (var sp = new SerialPort())
		            {
			            sp.PortName = comport; // /dev/ttyACM0 for linux - COM8 for Windows
			            sp.BaudRate = _settings.BaudRate; // 9600
			            sp.Open();
			            sp.WriteLine("Test string");
			            sp.Close();
		            }
	            }
	            catch (Exception e)
	            {
		            Console.WriteLine("Failed to communicate with display on port " + comport + ":");
		            Console.WriteLine(e);
	            }
            }
        }
    }
}

Danois90 · September 13, 2022, 11:33am

What do you understand by "Arduino"? Is it an Uno or a Nano or something else? Try to modify this in loop():

//while (!Serial.available()); //Do not do this

if (Serial.available())
{
  int bytesRead = Serial.readBytes(buffer, bufferSize - 1);
  buffer[bytesRead] = '\0';	
  log("Received " + String(bytesRead, DEC) + " bytes");
}

ReqPro1 · September 13, 2022, 11:37am

I have several Arduino models

The ones running this particular code are Uno and Mega 2560 - I do have Nano33 IoT but they are sporting a different microcontroller (SAMD21) and are not fully compatible with the TFT libraries I need for the TFT screen.

I have been running a similar sketch on the Nano 33 IoT with a LCD instead, and that works perfectly fine on Linux as well as Windows - the data is sent perfectly fine and displayed on the LCD....

johnwasser · September 13, 2022, 11:41am

Character in the Serial output buffer don't always get sent when a sketch crashes. I suggest adding "Serial.flush();" just after Serial.println() in log().

Danois90 · September 13, 2022, 11:45am

You should not use the "String" class / data type on AVR based MCU's like the Uno and Nano since that alone may cause crashes. Are you sure that the arduino crashes and reboots? Have you tried to blink the internal LED upon boot to be sure?

void setup()
{
  pinMode(LED_BUILTIN, OUTPUT);
  for (uint8_t i = 0; i < 3; i++)
  {
    digitalWrite(LED_BUILTIN, HIGH);
    delay(500);
    digitalWrite(LED_BUILTIN, LOW);
    delay(500);
  }
  ...
}

Will blink 3 times when booted.

Or comment out that Serial.println() since there is nothing that receives it in the other end. The serial connection is terminated after written to in C#, so the arduino is essentially writing to a closed connection.

ReqPro1 · September 13, 2022, 11:51am

Hi,

yes I am absolutely positive it reboots, as per the log functionality it writes to the log file whenever setup() is done being executed.

I can see in my logs that the entry "Setup complete." shows up again and again

I will add the blink functionality to make it more physically visible when it reboots instead of having to confirm by checking SD logs, just for comfort - thanks for the advice

Here's log output:

Setup complete.
Waiting for input...
Setup complete.
Waiting for input...
Setup complete.
Setup complete.
Waiting for input...
Setup complete.
Waiting for input...
Setup complete.

Danois90 · September 13, 2022, 12:04pm

Something just came to mind; Is the Arduino not being reset when a serial connection to it is opened? Maybe the serial connections on Windoze are persistent and on Linux they are not, that would explain the behavior..

Topic		Replies	Views
Weird issue with Serial.readBytes() Programming	15	1344	May 6, 2021
Windows serial unreliable - Linux 100% reliable Interfacing w/ Software on the Computer	23	5979	May 6, 2021
Serial read problem Programming	4	952	May 5, 2021
Bug Suggestions for the Arduino Project	2	1655	May 6, 2021
Serial communication (Windows to Arduino) Programming	5	2747	May 5, 2021

Trouble with Serial.readBytes on Windows vs Linux

Related topics