Pushing the Arduino Serial port to 1 MBps for fast DAQ system

Hello All,

I am currently trying to run an experiment that requires 10K samples/second of data transfered from the arduino (analog read) to my laptop. I have managed to compress the data I am sending to a maximum of 4 numbers and the \r\n...so 6 bytes of data per sample.

The fastest I have been able to get is using screen on my mac book pro (screen is a serial comm program) at a baud of 230400 bps. This should give me 230400 bps /(6bytes*8bits/byte) = 4800 samples/second, but I have only been able to achieve around 3300 samples/second. Fast...but not fast enough.

I have been scouring the forums and found this post

http://forum.arduino.cc/index.php/topic,61381.0.html

Where user robtillaart claims to be getting 2 Mbps data transfers. From my understanding, all that is required is to set the arduino baud to 1000000 and then the computer baud to 1000000. I understand there could be a large error rate, but that does not matter too much for the data I am receiving. The arduino runs at 16 MHz and my computer processing speed is 2.8GHz making 1 MBps an integer multiple of both clocks.

The problem is that I cannot seem to find a serial port interface on the computer side that will allow me to receive data at anything faster than 230400 bps. I have tried screen, minicom, MATLAB. Minicom and Matlab don't allow me to set baud rates above 230400 and screen gives garbage above 230400 bps.

I am looking for any help into this matter. I am also not limited to serial, it's just the only comm I know of. If anyone has any other better ideas Please let me know. The hardware I am working with is:

Mac Book Pro 2009
Arduino 2560MEGA

and am simply analog reading a voltage signal that is limited between 0 and 1.5 V

I used a PC with a terminal programm called putty.exe . The mac has also terminal programs that can capture to file.

another solution might be to write your own receive program (not tried but Python might do the job)

10K samples/second of data transfered from the arduino (analog read)

analogRead can do ~8100 samples per second leaving no time to communicate with the MAC. unless at lower precision??

Are you using analogRead() to get the 4 numbers? If yes, each read costs 0.1ms so you can't do any better than 3000 per second for all 4 channels. If you are using external ADC, then it depends on the ADC and serial port speed.

Hey Thanks! I managed to get it working using Realterm.exe. And as you guys have said, I realized my issue is not comms but reading the data. In fact, I found it took about 84 us to send 1 character over serial. 10 characters took 840 us. This is using serial.print. Is there a better way of sending data?

My other option is to use an external ADC communicating through SPI. Its a AD7942. It can handle up to 250K samples per second and has 14 bit resolution. If I hook this up using SPI do you think I will be able to get the data to my arduino and then to my computer at rates around 10K samples per second? Or does SPI take too much processing power? Later today I am going to try and get the SPI working to test it out but I haven't had any luck yet. The SPI on the AD7942 doesn't seem to work like other SPI devices, at least according to the data sheet.

I need to see the data in real time (or as close to that as possible) so storing to an SD card is not really an option ATM.

Is there a better way of sending data?

you can send it in binary format, that is only 2 bytes per value, and you can use compression to improve transport bandwidth.
Run-length compression is often a good compromis.

I wonder if there are any sort of higher speed TTL USB adapter that can do SPI. There is another problem hopefully not bothering you. The TTL USB adapter only sends the bytes to PC when its buffer is filled so you will have to assume the data were taken at fixed interval instead of using PC time stamp of when the data was received. They are received one chunk at a time instead of one byte at a time.

Another thing worth thinking about is the Leonardo. I wonder how fast its serial port is, with its USB capability.

liudr:
I wonder if there are any sort of higher speed TTL USB adapter that can do SPI.

Sure!

BUSpirate

TIAO USB Multi Protocol Adapter
http://www.diygadget.com/tiao-usb-multi-protocol-adapter-jtag-spi-i2c-serial.html

robtillaart:
you can send it in binary format, that is only 2 bytes per value, and you can use compression to improve transport bandwidth.
Run-length compression is often a good compromis.

Yes, I tried this but unsucessfully. To send my analog read values as binary I tried

Serial.print(voltage,BIN);

where voltage is the analog value. But I found I got slower data rates because the serial would send 1's and 0's as 8 bit characters, so it never really sent binary. How do I make it just send the binary data?

zoomx:

liudr:
I wonder if there are any sort of higher speed TTL USB adapter that can do SPI.

Sure!

BUSpirate
Bus Pirate - Wikipedia
TIAO USB Multi Protocol Adapter
http://www.diygadget.com/tiao-usb-multi-protocol-adapter-jtag-spi-i2c-serial.html

That looks interesting. How would I program it? Do I interface them with the arduino or directly with the SPI device (ADC)?

I didn't get around to working on the SPI stuff today but maybe on tuesday of next week I'll will be able to look at it.

Directly with the SPI device (ADC).

That is not how you send binary. You should use serial.write to write one byte at a time so an integer takes two lines

Serial.write(lowByte(number));
Serial.write(highByte(number));

Girino - Fast Arduino Oscilloscope

The author explain a technique for fast acquiring data. But he used only 8 bits.