Go Down

Topic: [SOLVED] Erraneous transmission at high baud rate for serial communications (Read 2310 times) previous topic - next topic

yw5aj

Dear all,

On my Arduino Uno, when I set the baud rate at 1,000,000, it worked still great. However the PC is often receiving entangled strings sent from the serial port due to some missing bytes. I guess because it is too high a rate?

For example, I am sending an integer (0-1023) line by line, so it should be:
"51\r\n"
"233\r\n"
"37\r\n"
"166\r\n"

And sometimes, the PC just missed one or two characters. For example, instead of "37\r\n" followed with "11\r\n", it will get "3\r11\r\n" or even "3711\r\n"!

Since this is sensor data, I can throw away some points if I know they are bad points... However there is also no easy & clean way that I can think of to identify these "bad lines".

Any thoughts?

Thanks!

Shawn

Coding Badly

I guess because it is too high a rate?
What is between your Arduino and your computer?  If it is just a USB cable then the high baud rate is not the problem.

The culprit is very likely the computer software.  I suggest using a known-to-be-reliable terminal application.

Quote
Since this is sensor data, I can throw away some points if I know they are bad points... However there is also no easy & clean way that I can think of to identify these "bad lines".
Include a CRC.


yw5aj

Thank you for your input! It is really helpful to know that high baud rate shouldn't be a problem. And yes, I am using a USB cable so the speed doesn't sound like a real problem.

I am using Python's pyserial to read the data. Specifically, I did Serial.println(val) to keep outputting an int, and used ser.readline() to keep reading a line.

Could it be the readline() function? Sorry about being naive on the serial communication, but I do see this line on pyserial documentation:

Quote
Note that when the serial port was opened with no timeout, that readline() blocks until it sees a newline (or the specified size is reached)
It shouldn't matter, right? The blocking should mean that no other devices can read the serial port, but arduino should still be able to write to it?



For the reference, below is the complete code:

In Python, reading the data:
Code: [Select]

import serial
import time

if __name__ == '__main__':
    ser = serial.Serial('COM3', baudrate=1000000)
    data = []
    time0 = time.time()
    while (time.time() - time0 < 5):  # Read data for 5 seconds
        data.append(ser.readline())
    ser.close()


In Arduino:
Code: [Select]

#define FASTADC 1  // Flag for prescale 16
// Code pasted for modifying ADCSRA
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

void setup() {
  Serial.begin(1000000);
  #if FASTADC
    // set prescale to 16
    sbi(ADCSRA,ADPS2) ;
    cbi(ADCSRA,ADPS1) ;
    cbi(ADCSRA,ADPS0) ;
  #endif
}

void loop() {
  int val;
  val = analogRead(A0);
  Serial.println(val);
}

Robin2

Reading data for 5 seconds does not make any sense.

You need to have start and end markers so the Python program knows when the data starts and end.

Look at this Python - Arduino demo (which is sticky at the top of this section ! )

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

yw5aj

Reading data for 5 seconds does not make any sense.

You need to have start and end markers so the Python program knows when the data starts and end.

Look at this Python - Arduino demo (which is sticky at the top of this section ! )

...R
Thank you Robin for your response! I did read your code before you post, but I thought that was a different kind of data being read. Mine was continuous data streaming from the sensor, with delimiters ("\r\n"). So reading the data for 5 seconds is a task that I need to do, without specifying a start and an end - or, the start and the end is the delimiters.

But I do think I should probably take a closer look to see whether there are concepts that I can convert to do the continuously reading task. Thanks again :)

Coding Badly


The first line is very likely sheared.  Do you throw out the first line in your post processing?


yw5aj

Yes I did throw the first data point away. The evils are in the middle. For example, if I plot my data, it look like this:



Where those spikes are incomplete data force-converted to int.

yw5aj

Actually - it might be the high baud rate! With more careful search, I found these posts:

http://forum.arduino.cc/index.php?topic=131872.0

People mention that at high baud rates, errors occur.
 

**More interestingly!!**

If I just send sine wave data generated, the 500,000 baud rate works OK. But, if I read from the analog pin, using analogRead(A0), then errors suddenly start to occur!

Obviously, it is analogRead(A0) that is messing with the serial communication. Any ideas?


Shawn

yw5aj

I have an Arduino Uno Rev 3.

I have a force sensitive resistor as my sensor. Current baud rate: 1,000,000. It works now:

1) Perfectly if I am sending generated sine wave (a very small an amplitude with only peak magnitude of 1) + sensor output;
2) Erroneous if I am sending sensor readout alone.

The "sine wave + sensor" code and plot:

Code: [Select]

#define FASTADC 1  // Flag for prescale 16
// Code pasted for modifying ADCSRA
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

void setup() {
  Serial.begin(1000000);
  #if FASTADC
    // set prescale to 16
    sbi(ADCSRA,ADPS2) ;
    cbi(ADCSRA,ADPS1) ;
    cbi(ADCSRA,ADPS0) ;
  #endif
}

void loop() {
  int val;
  val = analogRead(A0);
  val += int(512 * (sin(float(millis()) * 0.01) + 1)) * 0.001;
  Serial.println(val);
}




For the code of sensor alone, just comment out the "val += int(...." line.



The weird spikes are actually caused by missing bytes (characters here). For example, instead of "37\r\n" followed with "11\r\n", it will get "3\r11\r\n" or even "3711\r\n"!

The Python code to read the data are the same:

Code: [Select]

import serial
import time


if __name__ == '__main__':
    ser = serial.Serial('COM3', baudrate=1000000)
    time0 = time.time()
    f = open('data.csv', 'w')
    ser.readline()
    while (time.time() - time0 < 10):
        try:
            data = int(ser.readline())
            current_time = time.time() - time0
            f.write('%.8f, %d\n' % (current_time, data))
            f.flush()
        except ValueError:
            pass
    ser.close()
    f.close()


    # %% Plot data
    import matplotlib.pyplot as plt
    import numpy as np
    time_array, data_array = np.loadtxt('data.csv', delimiter=',').T
    plt.plot(time_array, data_array)
    plt.ylim(-100, 1500)
    plt.xlabel('Time (s)')
    plt.ylabel('Data (0-1023)')
    plt.tight_layout()
    plt.savefig('data_sensor_alone.png')

yw5aj

Actually - it might be the high baud rate! With more careful search, I found these posts:

http://forum.arduino.cc/index.php?topic=131872.0

People mention that at high baud rates, errors occur.
 

**More interestingly!!**

If I just send sine wave data generated, the 500,000 baud rate works OK. But, if I read from the analog pin, using analogRead(A0), then errors suddenly start to occur!

Obviously, it is analogRead(A0) that is messing with the serial communication. Any ideas?


Shawn
I just realized that this is indeed a different issue than the OP. I'll open a new thread then!

Here it is:  http://forum.arduino.cc/index.php?topic=316446.0


Coding Badly

The weird spikes are actually caused by missing bytes (characters here).
Why does the generated sine wave not have spikes?


yw5aj

Why does the generated sine wave not have spikes?


The "wave" you saw is my finger pressing on it. The generated sine wave, as you can see in the code, is only having peak magnitude of 1, which is trivial compared to the sensor input.

Good question - sorry that I did not made it more clear in the OP. I will edit it now.

Coding Badly

I just realized that this is indeed a different issue than the OP.
By your own admission, the topics are the same (dropped data).



yw5aj

I found an easier way to repeat the problem.

Code: [Select]

void setup() {
  Serial.begin(2000000);
}

void loop() {
  Serial.println(234);
}



Keep sampling with your favorite tool for 10,000 data points - there will be spikes.

Shawn

Go Up