3 Mbaud/Megabaud

Forum 2005-2010 (read only) General Exhibition
1

This is a bit of an exhibition, hardware question, and software question. I have a working 3Mbaud connection.

First the questions:
Is anyone running the Arduino at megabaud speeds? How many serial-frames do other people lose? I seem to lose about 0.05% of my frames even at low normal speeds...

Exhibition:

I get about 296kbytes/sec through this 3megabaud connection.

To get the 3Mbaud connection:

  • modify the EEPROM on the FTDI chip to put out a 12Mhz signal on pin 14
  • divide that frequency to get a 3Mhz signal
  • connect the 3Mhz signal to digital pin 4
  • set the arduino to slave synchronous mode
    I made the frequency divider with a 74HC595 and a 74HC14 I had laying around. I reprogrammed the FTDI chip using my Mac and the examples in the D2XX Driver Package http://www.ftdichip.com/Drivers/D2XX.htm . To get the Arduino into slave synchronous mode I used:
        Serial.begin(1000000);
        UCSR0C = (1<<UCSZ00) | (1<<UCSZ01) | (1<<UMSEL00) | (1<<UCPOL0);
2

I figured out why my error rate was extremely high(stupid programming problem), but I'm still losing about 0.0001% of the frames/bytes.

3

Hi!
Thanks for the information.
Can I have your actual baud?
Can you transfer 300KB/s with your synchronized setup?
How much is your "actual" transfer rate?
Thanks.

4

Yep, I get almost 300k/sec with this setup, but since the error rate is somewhat high, you have to have some kind of error checking / resend (I get a corrupt byte about every 10000 bytes). If a small error is OK, then you can just use it as a 300k/s connection.

I picked up an Arduino UNO at the Maker Faire last Saturday, but haven't had much free time to push it yet. The UNO should be able to do a LOT more than this without any hardware modification.

5

Do you really need all that speed?
Do you know that you send 8 bits but the usart as to generate 10 bits to create the start and stop bits?
You cant really acquire or even produce all that amount of data using the arduino.

6

You can push the bandwidth a little further by using 9 data bits:

  • 1 start
  • 9 data bits
  • 1 stop bit

Have the FTDI adapter accept the 9th bit as the parity bit and check if there was an error on the frame on your computer... but this is a pain in the ass.

(3000000/10) = 293k/s
(3000000/11)*(9/8) = 299.6k/s

I was using this to just grab data from the analog inputs. The analog inputs are meant to run at 125khz, but you can clock them at up to 8Mhz on a 16Mhz clocked chip. At 8Mhz, the analog inputs have about 2bits of useful data rather than the 10bits they are supposed to have.

7

Little bit offtopic... i know, sorry for that....

I wonder a little bit.... Datatransfer is on of the interessting things...
your attempt is impressive... i like it very much....

But, there are tons of sketches... tons of libraries...
but if you would serach for datacompession things... there are some advices but nothing else.

If i got transferspeed x with zero compression... there is a potiential to optimize content to speed up that.

Nobody does compression, so it seems to me....

Ah, "Atmega isn't able because of low clock and low ram...."

mmh, i don't think so, because most of the transfered Data is at an
simple scheme. Sensor data, Textdata.... mostly structured and you know what will happen.

So, one possibility is a simple Lookuptable on both communicating harware parts. (eg. Computer/Arduino or Arduino/Arduino).

I think hardware optimization like you do, is the first step.
But why nobody develop a simple compression protocol....

Why there isn't a simple rle or lza etc. library...?

If you discuss about +- 1 Mbaud... i think this is the second step... to speed up....

With 020 there are new String-Commands... what makes this simplier....

Maybe, stop tuning that +- Mbaud things and improve Dataquality/Size...

I am shure, that a compression-libary is possible...
to speed up further...

Greetings ChrisS

8

You can compress data, using simple algorithms, like the huffman

9

Tanks, i know... and with 7zip/rar also :wink:

Problem is, that there is no library... that i meant in my previous post.

Greetings ChrisS

10

If you are using 3/4Mbaud, then you REALLY don't have many CPU cycles.

Running at 3Mbaud sending one byte every frame on a 16Mhz CPU means you have to copy a byte into the serial buffer every 53.3 cpu cycles. It takes quite a few of those cycles just to check if the serial data buffer is empty and copy a new byte into the buffer... not to mention that you have to actually obtain the data from somewhere. This goes up to 58.6 cpu cycles for 9bit data frames, but you also have to spend some of those to set the 9th bit.

11

I bet that sending uncompressed is much more speedier than using Huffman...

12

Bet!

http://www.das-labor.org/wiki/AVR-Huffman/en
This should work if someone finde to make it communityready...

Ok.... know, somebody has done some comparsionable work...
http://www.mikrocontroller.net/topic/96157

Quote:"
Zur Geschwindigkeit:
bei 16MHZ dauert es eine 240x64 Pixel Grafik zu dekodieren mit 24%
kompressionsrate im 8bit Modus etwa 15ms
bei 4bit (30% gleiche Datei) etwa 19,5ms"

Ok, because most of you are speaking english....
A by 24% compressed picture within 30% similar data is decompressed in 19,5ms....

Huffman....

If you buffer data, compess, send and decompress.... the datarate would be the same...... (( <<< i win the bet :wink: ^^ ))

But you are able to transfer more data in the same time... OR you are faster with transmission, and lost som ms for compression and decompression...

Give it a try...

hopefully someone is able to adapt the already finished work for arduino...

But this is not a compression thread...

Its related if you send one Byte (no compression is faster), or if you send 3 MB compressed to 512KB...(much faster with compression and lower transmission time)

I am sure... if you really need 3Mbaud... there is MUUUCH Data which you want to transfer... :wink: so compression is the better way....

Greetings ChrisS

13

Well... If your data is very random, then any lossless compression scheme will actually INCREASE the amount of data to be sent rather than decrease it... Compression is all about probabilities. If you are using lossless compression and the data can't be compressed, then you are SOL (you have to send the uncompressed data plus a flag saying it is uncompressed).

If you are really wanting to use compression to increase the bandwidth, then I would look at lossy compression schemes. Your data will suffer, but you can be guaranteed to not increase the amount of bits needing to be sent.

14

Youre right, but i think in real world experiences the worst case...
255 different characters in a 256Byte Transmitbuffer is not very usual.

In the most cases you will nearly know what come over the line...
mostly no binary datastreams of all characters mainly....

I think one indicating byte (Compression ON/OFF) doesn't blow up the whole transfers very much....

This indicating byte could be set short after buffer analysis... to avoid bad and blown transmitting of non compressable content...

In the most cases there is content compressable... related of the codec you use... huffman... mmh, long approved... there are better... but 16Mhz is our limit 2010 :wink:

Was only an idea, to speed up further...

Greetings ChrisS

15

For sending ADC data to PC/Client, compression it not good idea :slight_smile:
At least you will miss some samples due huffman search. It increases complexity and this also increases error rates. Also consumes much more "power" and "ram" and flash space...etc. Also simple is better :slight_smile: So I win the bet from this aspect :wink:

I don't think 16Mhz isn't limit. We can use 20Mhz Xtals easily and it's "fully" compatible, if you update your wiring.c file with this patch (I don't know why stuff don't add 20Mhz fix to latest library.)

It's also sad thing that "Uno" has 16Mhz too. I really don't understand/know why they resist on 16Mhz. Do they any valid reason to stick to 16Mhz?

gabebear, what is your highest serial async port speed? Mine capped at exactly 2Mbaud.

16 
At least you will miss some samples due huffman search.

Interrupt trigger? No.

It increases complexity and this also increases error rates.

you invented fire, but don't use it... you could burn your hands....
better use a stone as using an microcontroller... a stone has less errorrates... No. If there are errors... then in communication, not in math... so use a checksum, and there couldn't be errors... or use better wires, shielding, and drive ur hardware within the specs.

So no.

Hum?

tried a lot with 20Mhz the last days... but i can't get the bootloader to upload... rewired everything while modified bootloaderfiles to freq 20 and baudrate calculations... (.c) can't get it uploading... 16Mhz works fine... this problem is not related to wiring.c i think....

Greetings ChrisS

17

The fastest async setting is 1/8th the CPU_Mhz, so 2Mbaud for 16Mhz(2.5Mbaud for 20Mhz)... the problem is this isn't a setting on the FTDI serial adapter(I think only 1Mbaud and 3Mbaud are available). The fastest I've done is 1Mbaud async.

As far as missing samples... yes, it would be very easy to miss some samples while off doing other stuff. Triggers/interrupts aren't very good for what I was doing because they add extra cycles( ~4 cycles per interrupt ) to interrupt the code that is currently running and jump to the new code... then you have another 5 cycles to leave the interrupt. When you have 53 total cycles you don't want to waste several on an interrupt for serial and several on an interrupt for analog input.

18

ChrisS, you might be good on letters but when it comes to stable programs, simple is better. Yes we can use every feature of MCU for this job. A standard developer couldn't understand what I made with code even If I put lots of explanation string due complex nature of code for using every property of MCU... Do you wanted to try to understand such a code? I mean, you don't need fire for drink your cold soda. If you love drinking it on opposite of the fireplace you can, but it's not required for standard people and this will be add some cost for burning wooden for people just wanted to drink soda... :wink: Some says my grammar is bad but I think you understand what I trying to tell.

gabebear, What? I think you talking about ADC sampling, not RS232 sending capacity right?... I am just sending constant value in loop, so not sampling ADC, on test purposes and setup gives me exactly 2Mbps nothing more. I wanted to have little higher speed than actual ADC speed. Weird thing is windows only could give ~37KB/s. I try it on WinXP on Real HW and than on VM too. Nothing changed...

If I use prescaler value of 2 (0x1 or 0x0 for ADPS[2-0] ) I got nothing meaningful, but using prescaler 4 (0x2 for ADPS[2-0]) gives meaningful values. So it's okey for me 2Mbaud for now. But I wanted to have 3Mbaud on Async too.I think it's better to have 2,5Mbaud connection because of losses and overheads. At least to know why it doesn't support 2,5 and 3Mbaud. It's due driver or chip or ATmel... who knows.....

I think it's better to produce a good program that uses OpenGL to display data (Yes, it's really needed for displaying 200Kpsp).
Thanks.

Edit: Yeah, I look ATmega datasheet and see the truth, CPU cannot handle >2Mbaud with 16Mhz CPU. So you mean RS232 port, not ADC sampling. :slight_smile: So I need synchronized mod for 3Mbaud. Thanks. You save my life, I will try to understand for a weeks why my FTDI 232R cannot work > 2Mbaud :slight_smile: