[SOLVED] Erraneous transmission at high baud rate for serial communications

Actually - it might be the high baud rate! With more careful search, I found these posts:

http://forum.arduino.cc/index.php?topic=131872.0

People mention that at high baud rates, errors occur.

More interestingly!!

If I just send sine wave data generated, the 500,000 baud rate works OK. But, if I read from the analog pin, using analogRead(A0), then errors suddenly start to occur!

Obviously, it is analogRead(A0) that is messing with the serial communication. Any ideas?

Shawn

I have an Arduino Uno Rev 3.

I have a force sensitive resistor as my sensor. Current baud rate: 1,000,000. It works now:

  1. Perfectly if I am sending generated sine wave (a very small an amplitude with only peak magnitude of 1) + sensor output;
  2. Erroneous if I am sending sensor readout alone.

The "sine wave + sensor" code and plot:

#define FASTADC 1  // Flag for prescale 16
// Code pasted for modifying ADCSRA
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

void setup() {
  Serial.begin(1000000);
  #if FASTADC
    // set prescale to 16
    sbi(ADCSRA,ADPS2) ;
    cbi(ADCSRA,ADPS1) ;
    cbi(ADCSRA,ADPS0) ;
  #endif
}

void loop() {
  int val;
  val = analogRead(A0);
  val += int(512 * (sin(float(millis()) * 0.01) + 1)) * 0.001;
  Serial.println(val);
}

For the code of sensor alone, just comment out the "val += int(...." line.

The weird spikes are actually caused by missing bytes (characters here). For example, instead of "37\r\n" followed with "11\r\n", it will get "3\r11\r\n" or even "3711\r\n"!

The Python code to read the data are the same:

import serial
import time


if __name__ == '__main__':
    ser = serial.Serial('COM3', baudrate=1000000)
    time0 = time.time()
    f = open('data.csv', 'w')
    ser.readline()
    while (time.time() - time0 < 10):
        try:
            data = int(ser.readline())
            current_time = time.time() - time0
            f.write('%.8f, %d\n' % (current_time, data))
            f.flush()
        except ValueError:
            pass
    ser.close()
    f.close()


    # %% Plot data
    import matplotlib.pyplot as plt
    import numpy as np
    time_array, data_array = np.loadtxt('data.csv', delimiter=',').T
    plt.plot(time_array, data_array)
    plt.ylim(-100, 1500)
    plt.xlabel('Time (s)')
    plt.ylabel('Data (0-1023)')
    plt.tight_layout()
    plt.savefig('data_sensor_alone.png')

yw5aj:
Actually - it might be the high baud rate! With more careful search, I found these posts:

USB Serial Baud Rates? ... or other faster USB communication methods? - Arduino Due - Arduino Forum

People mention that at high baud rates, errors occur.

More interestingly!!

If I just send sine wave data generated, the 500,000 baud rate works OK. But, if I read from the analog pin, using analogRead(A0), then errors suddenly start to occur!

Obviously, it is analogRead(A0) that is messing with the serial communication. Any ideas?

Shawn

I just realized that this is indeed a different issue than the OP. I'll open a new thread then!

Here it is: http://forum.arduino.cc/index.php?topic=316446.0

yw5aj:
The weird spikes are actually caused by missing bytes (characters here).

Why does the generated sine wave not have spikes?

The "wave" you saw is my finger pressing on it. The generated sine wave, as you can see in the code, is only having peak magnitude of 1, which is trivial compared to the sensor input.

Good question - sorry that I did not made it more clear in the OP. I will edit it now.

yw5aj:
I just realized that this is indeed a different issue than the OP.

By your own admission, the topics are the same (dropped data).

Threads merged.

I found an easier way to repeat the problem.

void setup() {
  Serial.begin(2000000);
}

void loop() {
  Serial.println(234);
}

Keep sampling with your favorite tool for 10,000 data points - there will be spikes.

Shawn

Hi Coding Badly,

I feel that this post deviated far enough from the OP, as I have made progresses to narrow down the source of the problem. People will get lost if I keep posting here. Could you please help review whether the following post is OK to create a new one? It would be OK to say no too and I really appreciate your work in help organizing this forum.

Subject: Sending constant int from Arduino to PC causes error

I am using Arduino Uno Rev 3, and according to this link we can achieve 2000000 baud rate.

This problem can be easily repeated by the following code:

void setup() {
  Serial.begin(2000000);
}

void loop() {
  int val = 234;
  Serial.println(val);
}

I am using the serial-USB cable. To read the data, I used Python pyserial (< 20 lines of code including empty lines and import lines):

import serial
import matplotlib.pyplot as plt

LEN = 10000
data = []
ser = serial.Serial('COM3', baudrate=2000000)
for i in range(LEN):
    try:
        data.append(int(ser.readline()))
    except:
        pass
ser.close()

# %% Plot data
plt.figure(figsize=(6, 4))
plt.plot(data)
plt.savefig('data.png')

Here is what I got:

yw5aj:
Could you please help review whether the following post is OK to create a new one?

Really? Rather than helping you with your problem you want me to waste my time justifying my moderation decisions? I will pass.

Don't take me wrong please! Again, as I said in the post above, I really appreciate your time and help in organizing the forum threads. Instead of trying to fight, I was thinking whether that would be a better way to discuss this issue. I will not post a new thread then.

Please don't misunderstand me, who truly appreciate all your effort!

Shawn

I cannot get it to fail (excess 234s trimmed)...

C:\ >C:\Python34\python
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import serial
>>>
>>> LEN = 10000
>>> data = []
>>> ser = serial.Serial('COM3', baudrate=2000000)
>>> for i in range(LEN):
...     try:
...         data.append(int(ser.readline()))
...     except:
...         pass
...
>>> ser.close()
>>>
>>> data
[234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234,
 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234,
 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234,
 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234,
....
 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234,
 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234, 234]
>>>

...which brings us full circle to Reply #1. Your Python script is not removing data from the serial buffer fast enough. You need some combination of: faster PC, better kernel driver, faster Python run-time, different computer-side programming language, framing, error detection.

Thanks a lot for testing it Coding! I have two thoughts though,

  1. One reason that makes me think it is not a issue from slow computer/connection is my Reply #8, where I added a very small sinusoidal curve to the int, and it worked perfectly. It is still a high baud rate, so if the speed is an issue, it would fail there too. If you don't mind, could you please take a look at that thread's code? I find that phenomenon really weird to understand (looks like the data type was automatically promoted by adding a value to it, but then again it is still int).

  2. There is another slight chance (I might be totally wrong): because the error rate is relatively low, I only find out the "spikes" by plotting them (eye-balling them through the IDLE console does not work). If you do

for datum in data:
    if datum > 234:
        print(datum)

Would any value be printed out? (I might be wrong though)

Thanks again - I do appreciate your time into this very much!

Shawn

yw5aj:

  1. One reason that makes me think it is not a issue from slow computer/connection is my Reply #8, where I added a very small sinusoidal curve to the int, and it worked perfectly. It is still a high baud rate, so if the speed is an issue, it would fail there too.

You increased the amount of work the Arduino must do between println calls. That slows the data rate. It slows the data rate enough that your computer can keep up.

The code is functionally equivalent to this where PauseForSlowComputer is high enough that the computer can keep up...

#define FASTADC 1  // Flag for prescale 16
// Code pasted for modifying ADCSRA
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

void setup() {
  Serial.begin(1000000);
  #if FASTADC
    // set prescale to 16
    sbi(ADCSRA,ADPS2) ;
    cbi(ADCSRA,ADPS1) ;
    cbi(ADCSRA,ADPS0) ;
  #endif
}

const unsigned PauseForSlowComputer = 50;

void loop() {
  int val;
  val = analogRead(A0);
  delayMicroseconds( PauseForSlowComputer );
  Serial.println(val);
}

yw5aj:
2) There is another slight chance (I might be totally wrong): because the error rate is relatively low, I only find out the "spikes" by plotting them (eye-balling them through the IDLE console does not work). If you do ... Would any value be printed out? (I might be wrong though)

This has been running for about 10 minutes (precise capture of the screen)...

C:\windows\system32 >C:\Python34\python
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import serial
>>>
>>> count = 0
>>> ser = serial.Serial('COM3', baudrate=2000000)
>>> ser.flush()
>>> while True:
...   count = count + 1
...   line = ser.readline()
...   if line != b'234\r\n':
...     print( '#', count, ':', line )
...

You are totally right! My goodness I should have thought about that. Thank you so much for your helpful experience! Finally got it: it worked when the sinusoid curve is there because it added a wait time between writing the serial port!

And - thank you for running the test too! Really appreciate it.

I will now update the title to show it has been answered.

You are welcome.

yw5aj:
Thank you Robin for your response! I did read your code before you post, but I thought that was a different kind of data being read. Mine was continuous data streaming from the sensor, with delimiters ("\r\n").

So use \r\n as a delimiter in place of the <> that my code uses. In fact you can probably just use \n on its own.

The second example in serial input basics just uses an end-marker rather than start- and end-markers.

...R