I’m currently working on a fast data logging system using the built-in ADC of Arduino. One of my main concerns is ensuring that no data is lost during high-speed serial communication between Arduino and a PC.
To test this, I’m sending 1000 ADC readings from Arduino and reading them using Python (PySerial). This should help verify whether the serial communication can keep up without dropping data.
Arduino Code (Sending 1000 ADC Readings via Serial)
#define NUM_SAMPLES 1000
void setup() {
Serial.begin(115200); // High-speed baud rate
while (!Serial);
}
void loop() {
for (int i = 0; i < NUM_SAMPLES; i++) {
int adcValue = analogRead(A0);
Serial.println(adcValue);
}
while(1); // Stop after sending 1000 data points
}
Python Code (Reading Data from Arduino Using PySerial)
import serial
# Open serial connection (adjust port name as needed)
ser = serial.Serial('COM3', 115200, timeout=1)
received_data = []
num_samples = 1000
for _ in range(num_samples):
line = ser.readline().strip() # Read and remove trailing spaces/newlines
if line:
try:
received_data.append(int(line)) # Convert string to integer
except ValueError:
print("Invalid data received:", line)
ser.close()
# Check if all 1000 samples were received
print(f"Received {len(received_data)} out of {num_samples} samples.")
if len(received_data) == num_samples:
print("No data loss detected!")
else:
print(f"Data loss detected: {num_samples - len(received_data)} samples missing.")
Discussion Points
I’d like to discuss the most effective methods for achieving high-speed ADC data acquisition while maintaining data integrity. Some key aspects I’m considering:
Optimal baud rate to ensure reliable high-speed communication.
Binary vs. ASCII transmission (would sending raw binary reduce overhead?).
Buffering strategies to prevent data loss.
Alternative communication methods (SPI, I2C, or direct USB transfer).
If anyone has experience with fast real-time data logging on Arduino, I’d love to hear your thoughts! Have you encountered serial data loss at high speeds? What strategies did you use to overcome it?
You have you not mentioned error detection and correction and acknowledgement of messages to detect missing data? Those are all things that are usually used to ensure data integrity.
assuming the UNO ADC can convert 1000 samples/second then transmitting binary data at a high serial baud project should be possible
you would need to transmit data frames with error checking etc
have a look at SerialTransfer library
the UNO ADC is 10bits - you could send a sample in two bytes or do some compression
you need to experiment!
This is somewhat open-ended and difficult to answer. It depends on what you mean by "fast", i.e. we need a number. Also "an Arduino" is too generic, which board?
However, in general to get "fast" and reliable you need
a high bit rate medium
fast CPU at each end
an efficient encoding scheme
use blocks to transfer data to reduce overhead
use a sequence number to identify packets
use a CRC to verify integrity
have a flow control method
Usually, when people try to roll their own protocol it's either has fatal flaws or ends up re-inventing TCP. It is usually better to adapt an existing protocol.
115200 is not really a high-speed baud rate... I recently worked on a product which used a 1 Mega-bit rate, but that is only "high" for a serial connection. Gigabit ethernet is now fairly common.
the term high speed does depends on context
some time ago worked on a project reading two ADCs each at 5Msamples/second transmitting results over Ethernet to a PC to be processed by Matlab
the dev boards which were based on Texas DSPs cost over £2K each plus another £2K for the ADC daughter board
Configure both sides for that baud rate and let 'er rip
You're not going to run out of memory: the max serial buffer on a PC is huge. Not as massive as the network buffering, but big enough that it shouldn't be an issue.
Unless you are trying to transfer this data many hundreds of feet, data corruption is unlikely.
If you run out of serial transfer speed, then I'd think about using an arduino that supports USB HID natively. That should be much faster, IIRC.
