Go Down

Topic: 3 Mbaud/Megabaud (Read 2804 times) previous topic - next topic

gabebear

This is a bit of an exhibition, hardware question, and software question. I have a working 3Mbaud connection.

First the questions:
   Is anyone running the Arduino at megabaud speeds? How many serial-frames do other people lose? I seem to lose about 0.05% of my frames even at low normal speeds...

Exhibition:

I get about 296kbytes/sec through this 3megabaud connection.

To get the 3Mbaud connection:

  • modify the EEPROM on the FTDI chip to put out a 12Mhz signal on pin 14
  • divide that frequency to get a 3Mhz signal
  • connect the 3Mhz signal to digital pin 4
  • set the arduino to slave synchronous mode
I made the frequency divider with a 74HC595 and a 74HC14 I had laying around. I reprogrammed the FTDI chip using my Mac and the examples in the D2XX Driver Package http://www.ftdichip.com/Drivers/D2XX.htm . To get the Arduino into slave synchronous mode I used:
Code: [Select]
       Serial.begin(1000000);
       UCSR0C = (1<<UCSZ00) | (1<<UCSZ01) | (1<<UMSEL00) | (1<<UCPOL0);



gabebear

I figured out why my error rate was extremely high(stupid programming problem), but I'm still losing about 0.0001% of the frames/bytes.

E.U.A.

Hi!
Thanks for the information.
Can I have your actual baud?
Can you transfer 300KB/s with your synchronized setup?
How much is your "actual" transfer rate?
Thanks.

gabebear

Yep, I get almost 300k/sec with this setup, but since the error rate is somewhat high, you have to have some kind of error checking / resend (I get a corrupt byte about every 10000 bytes). If a small error is OK, then you can just use it as a 300k/s connection.

I picked up an Arduino UNO at the Maker Faire last Saturday, but haven't had much free time to push it yet. The UNO should be able to do a LOT more than this without any hardware modification.

Senso

Do you really need all that speed?
Do you know that you send 8 bits but the usart as to generate 10 bits to create the start and stop bits?
You cant really acquire or even produce all that amount of data using the arduino.

gabebear

You can push the bandwidth a little further by using 9 data bits:
  • 1 start
  • 9 data bits
  • 1 stop bit

Have the FTDI adapter accept the 9th bit as the parity bit and check if there was an error on the frame on your computer... but this is a pain in the ass.

(3000000/10) = 293k/s
(3000000/11)*(9/8) = 299.6k/s

I was using this to just grab data from the analog inputs. The analog inputs are meant to run at 125khz, but you can clock them at up to 8Mhz on a 16Mhz clocked chip. At 8Mhz, the analog inputs have about 2bits of useful data rather than the 10bits they are supposed to have.

ChrisS

Little bit offtopic... i know, sorry for that....

I wonder a little bit.... Datatransfer is on of the interessting things...
your attempt is impressive... i like it very much....

But, there are tons of sketches... tons of libraries...
but if you would serach for datacompession things... there are some advices but nothing else.

If i got transferspeed x with zero compression... there is a potiential to optimize content to speed up that.

Nobody does compression, so it seems to me....

Ah, "Atmega isn't able because of low clock and low ram...."

mmh, i don't think so, because most of the transfered Data is at an
simple scheme. Sensor data, Textdata.... mostly structured and you know what will happen.

So, one possibility is a simple Lookuptable on both communicating harware parts. (eg. Computer/Arduino or Arduino/Arduino).

I think hardware optimization like you do, is the first step.
But why nobody develop a simple compression protocol....

Why there isn't a simple rle or lza etc. library...?

If you discuss about +- 1 Mbaud... i think this is the second step... to speed up....

With 020 there are new String-Commands... what makes this simplier....

Maybe, stop tuning that +- Mbaud things and improve Dataquality/Size...

I am shure, that a compression-libary is possible...
to speed up further...

Greetings ChrisS
Sturmfabrik - mediale Dienstleistungen
www.sturmfabrik.de

Senso

You can compress data, using simple algorithms, like the huffman
http://en.wikipedia.org/wiki/Huffman_coding

ChrisS

Tanks, i know... and with 7zip/rar also ;)

Problem is, that there is no library... that i meant in my previous post.

Greetings ChrisS
Sturmfabrik - mediale Dienstleistungen
www.sturmfabrik.de

gabebear

If you are using 3/4Mbaud, then you REALLY don't have many CPU cycles.

Running at 3Mbaud sending one byte every frame on a 16Mhz CPU means you have to copy a byte into the serial buffer every 53.3 cpu cycles. It takes quite a few of those cycles just to check if the serial data buffer is empty and copy a new byte into the buffer... not to mention that you have to actually obtain the data from somewhere. This goes up to 58.6 cpu cycles for 9bit data frames, but you also have to spend some of those to set the 9th bit.

E.U.A.

I bet that sending uncompressed is much more speedier than using Huffman...

ChrisS

#11
Oct 07, 2010, 03:23 pm Last Edit: Oct 07, 2010, 03:29 pm by ChrisS Reason: 1
Bet!

http://www.das-labor.org/wiki/AVR-Huffman/en
This should work if someone finde to make it communityready...

Ok.... know, somebody has done some comparsionable work...
http://www.mikrocontroller.net/topic/96157

Quote:"
Zur Geschwindigkeit:
bei 16MHZ dauert es eine 240x64 Pixel Grafik zu dekodieren mit 24%
kompressionsrate im 8bit Modus etwa 15ms
bei 4bit (30% gleiche Datei) etwa 19,5ms"

Ok, because most of you are speaking english....
A by 24% compressed picture within 30% similar data is decompressed in 19,5ms....

Huffman....

If you buffer data, compess, send and decompress.... the datarate would be the same...... (( <<< i win the bet ;) ^^ ))

But you are able to transfer more data in the same time... OR you are faster with transmission, and lost som ms for compression and decompression...

Give it a try...

hopefully someone is able to adapt the already finished work for arduino...

But this is not a compression thread...


Its related if you send one Byte (no compression is faster), or if you send 3 MB compressed to 512KB...(much faster with compression and lower transmission time)

I am sure... if you really need 3Mbaud... there is MUUUCH Data which you want to transfer... ;) so compression is the better way....

Greetings ChrisS
Sturmfabrik - mediale Dienstleistungen
www.sturmfabrik.de

gabebear

#12
Oct 07, 2010, 04:06 pm Last Edit: Oct 07, 2010, 04:07 pm by gabebear Reason: 1
Well... If your data is very random, then any lossless compression scheme will actually INCREASE the amount of data to be sent rather than decrease it... Compression is all about probabilities. If you are using lossless compression and the data can't be compressed, then you are SOL (you have to send the uncompressed data plus a flag saying it is uncompressed).

If you are really wanting to use compression to increase the bandwidth, then I would look at lossy compression schemes. Your data will suffer, but you can be guaranteed to not increase the amount of bits needing to be sent.

ChrisS

Youre right, but i think in real world experiences the worst case...
255 different characters in a 256Byte Transmitbuffer is not very usual.

In the most cases you will nearly know what come over the line...
mostly no binary datastreams of all characters mainly....

I think one indicating byte (Compression ON/OFF) doesn't blow up the whole transfers very much....

This indicating byte could be set short after buffer analysis... to avoid bad and blown transmitting of non compressable content...

In the most cases there is content compressable... related of the codec you use... huffman... mmh, long approved... there are better... but 16Mhz is our limit 2010 ;)

Was only an idea, to speed up further...

Greetings ChrisS
Sturmfabrik - mediale Dienstleistungen
www.sturmfabrik.de

E.U.A.

#14
Oct 07, 2010, 04:57 pm Last Edit: Oct 07, 2010, 04:59 pm by E.U.A.. Reason: 1
For sending ADC data to PC/Client, compression it not good idea :)
At least you will miss some samples due huffman search. It increases complexity and this also increases error rates. Also consumes much more "power" and "ram" and flash space...etc. Also simple is better :) So I win the bet from this aspect ;)

I don't think 16Mhz isn't limit. We can use 20Mhz Xtals easily and it's "fully" compatible, if you update your wiring.c file with this patch (I don't know why stuff don't add 20Mhz fix to latest library.)

It's also sad thing that "Uno" has 16Mhz too. I really don't understand/know why they resist on 16Mhz. Do they any valid reason to stick to 16Mhz?

gabebear, what is your highest serial async port speed? Mine capped at exactly 2Mbaud.

Go Up