Problems sending serial data more than about 180 bytes

I’m having weird problems sending serial data over USB to my Arduino Uno R3. I can send around 180 ish bytes at a time fine, but any more than that and it looks like data starts getting lost. I’ve put together a basic sketch and Windows program which demonstrates the problem I’m having.

Sketch:

int CurrentValueCounter;

void setup() {
  Serial.begin(115200);
  CurrentValueCounter = 0;
}

void loop() {
  while (Serial.available() != 0)
  {
    int incomingByte = Serial.read();
    Serial.print(String(incomingByte) + " ");
    ++CurrentValueCounter;
    if(incomingByte != 0)
    {
      Serial.println("");
      Serial.println("Received counter " + String(incomingByte) + " after " + String(CurrentValueCounter) + " bytes");
      CurrentValueCounter = 0;
      Serial.println("");
    }
  }
}

I’ve tried both a C# application and a C++ one to send the data.

C#:

using System;

namespace Sender
{
    class Program
    {
        static System.IO.Ports.SerialPort outputComPort;
        static int counter = 1;
        static int delay = 0;

        static void Main(string[] args)
        {
            outputComPort = new System.IO.Ports.SerialPort("COM11", 115200, System.IO.Ports.Parity.None, 8, System.IO.Ports.StopBits.One);
            outputComPort.Handshake = System.IO.Ports.Handshake.None;
            outputComPort.NewLine = "\r\n";
            outputComPort.DataReceived += OutputComPort_DataReceived;
            outputComPort.Open();

            System.Timers.Timer timer = new System.Timers.Timer();
            timer.Elapsed += Timer_Elapsed;
            timer.Interval = 10000;
            timer.Enabled = true;

            Console.ReadKey();
        }

        private static void Timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            if(delay > 0)
            {
                --delay;
                return;
            }

            Console.WriteLine();
            Console.WriteLine("Sending Counter: " + counter);
            byte[] data = new byte[192];
            data[data.Length - 1] = (byte)counter;

            outputComPort.Write(data, 0, data.Length);

            counter = (counter + 1) % 255;
        }

        private static void OutputComPort_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
        {
            while(outputComPort.BytesToRead > 0)
            {
                byte[] buffer = new byte[100];
                int numBytesRead = outputComPort.Read(buffer, 0, 100);
                String input = System.Text.Encoding.Default.GetString(buffer, 0, numBytesRead);
                Console.Write(input);
            }
        }
    }
}

C++:

#include "stdafx.h"
#include <windows.h>
#include <string>
#include <iostream>
#include <ctime>

int main()
{
	// Open serial port
	HANDLE serialHandle = CreateFile(L"\\\\.\\COM11", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);

	// Do some basic settings
	DCB serialParams = { 0 };
	serialParams.DCBlength = sizeof(serialParams);

	GetCommState(serialHandle, &serialParams);
	serialParams.BaudRate = 115200;
	serialParams.ByteSize = 8;
	serialParams.StopBits = 1;
	serialParams.Parity = 0;
	SetCommState(serialHandle, &serialParams);

	// Set timeouts
	COMMTIMEOUTS timeout = { 0 };
	timeout.ReadIntervalTimeout = MAXDWORD;
	timeout.ReadTotalTimeoutConstant = 0;
	timeout.ReadTotalTimeoutMultiplier = 0;
	timeout.WriteTotalTimeoutConstant = 50;
	timeout.WriteTotalTimeoutMultiplier = 50;

	SetCommTimeouts(serialHandle, &timeout);

	bool quit = false;
	HANDLE consoleHandle = GetStdHandle(STD_INPUT_HANDLE);
	INPUT_RECORD inputRecord;
	DWORD charCode;
	char readBuffer[255];
	char writeBuffer[192];
	int writeBufferLength = ARRAYSIZE(writeBuffer);
	memset(writeBuffer, 0, writeBufferLength);
	clock_t prevTime = clock();
	int counter = 1;
	while (!quit)
	{
		DWORD numConsoleEvents = 0;
		GetNumberOfConsoleInputEvents(consoleHandle, &numConsoleEvents);
		if (numConsoleEvents)
		{
			ReadConsoleInput(consoleHandle, &inputRecord, 1, &charCode);
			if (inputRecord.EventType == KEY_EVENT && ((KEY_EVENT_RECORD&)inputRecord.Event).bKeyDown)
			{
				quit = true;
			}
		}

		DWORD numBytesRead = 0;
		if (ReadFile(serialHandle, readBuffer, 255, &numBytesRead, NULL))
		{
			if (numBytesRead)
			{
				std::cout << std::string(readBuffer, numBytesRead);
			}
		}

		if (((clock() - prevTime) / CLOCKS_PER_SEC) >= 10)
		{
			std::cout << std::endl << "Sending counter: " << counter << std::endl;

			writeBuffer[writeBufferLength - 1] = counter;
			int numBytesToWrite = writeBufferLength;
			DWORD numBytesWritten = 0;
			while (numBytesToWrite > 0 && WriteFile(serialHandle, writeBuffer, numBytesToWrite, &numBytesWritten, NULL))
			{
				numBytesToWrite -= numBytesWritten;
			}

			++counter;

			prevTime = clock();
		}
	}

	CloseHandle(consoleHandle);
	CloseHandle(serialHandle);

    return 0;
}

Basically, the Windows program will just send an array of bytes, with the last entry in the array set to a counter value. The Arduino sketch echos back the bytes as they’re received, and also prints a line when it receives a non 0 byte. I’ve tried running at 9600 (making sure the Arduino and Windows app always match their baud rate), but I still get problems. An example output from the C++ app running at 115200, trying to send an array of 192 bytes is attached (Broken.txt).

I also get some garbage when the port is first opened but only when running at 115200.

Running the same code, but only sending 160 bytes at a time, gives me solid results (OK.txt)

I’m waiting 10 seconds between sending each array, so I don’t think I’m overloading anything, and the Arduino isn’t doing anything else, so should be able to process the incoming data.

Is it right that I can only send somewhere in the region of 180-ish bytes in one go without data being corrupted? This seems like a really small amount.

Has anyone come across anything like this before? I’m using Arduino 1.8.5 (Windows Store 1.8.10.0) if that makes any difference?

Thanks

Broken.txt (1.73 KB)

OK.txt (1.55 KB)

Some say to avoid a String object in the Arduino Uno. I say: Use the String objects in ARM processors (Arduino Zero, Arduino Due), but not in a Arduino Uno.

This "Serial.print(String(incomingByte) + " ")" creates a String object; allocates memory in the heap; converts the byte; sends the object to Serial.print; convert it back to normal characters; transmit them via serial. A few lines further, you do that again, twice! It is amazing that you can do 180 bytes that way :o

The problem is not reading the bytes with Serial.read. The problem is so many Serial.print and the use of the String objects.

When the Arduino Uno is reading data from 115200, collect the bytes in buffer, or do some math on them (not too much) or output them to pins, or whatever. If you want to return something while receiving the data, then try to send just one byte for each byte received. No more.

You're using the String class on a microcontroller with limited memory and no garbage collection. That is known to cause issues, especially when you are using the += operator to concatenate strings. That shoots your heap all full of holes like swiss cheese. String class is no good on a microcontroller for anything more than simple demonstration code. Learn to use char arrays and handle strings the old fashioned way and things will probably work better.

Oh man, I feel a bit silly now :) my original problem was that I was losing serial data, so I put in some debug logging to try and figure out what was going on. The I boiled it down to what I had above and still had the problems. So my original code was probably just doing too much, and my test code above is doing too much too. I rewrote my test code to not use the string stuff and just print one thing at the end of each byte array and it works fine and fast. So I just need to rework my original code to not do anything while it's receiving data and I should be good. Thanks for the help :)

On a side note - does anyone know what's going on with the garbage that gets read by the desktop when the port is first opened? Only seems to happen at baud >= 115200.

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data.

If you change the value of numChars in the 2nd or 3rd example you can receive as much data as you want.

…R

Try a Leonardo board, or (Pro) Micro or ARM board like the Arduino Zero, M0, Due. They have the serial port implemented in the processor as a USB CDC device. The baudrate is not really used and data can go very fast over the USB cable.

gogorobot: On a side note - does anyone know what's going on with the garbage that gets read by the desktop when the port is first opened? Only seems to happen at baud >= 115200.

You can try a short delay in setup; 100 ms or so.

I've seen it as well (never related it to baudrate though) but it always seemed to help.