Go Down

Topic: Serial protocol how-to (Read 2 times) previous topic - next topic

melka

Hey there

I have a processing sketch that loads a black/white 8*8px animated gif, analyze each frames and sends the value corresponding to the lit pixels (8 bytes, one per line) to the arduino board, approximately 25 times per second (maybe more). On the arduino side, I have a max7221 hooked up to a 8*8 matrix.

After a few sends, my data gets f****ed up, certainly because I don't have a proper serial protocol. I can't use libraries like firmata or SMS, because ultimately, this prototype sketch I'm doing for a friend will be recoded on a PIC mcu.

How would you implement a communication protocol that can run for approximately 2 hours at (let's say) 50 frames per second (3200bauds of pure data, without the header and stuff for the protocol) without any bug ? ^^

Thanks for any infos

melka
http://melka.one.free.fr/blog/
http://www.flickr.com/photos/melkaone/

melka

You know what ? I think I got it. On the arduino side, I just make sure that I have 8 byte in the buffer before reading it, then I flush. I clear the Processing buffer after 8 sends too. Seems to work.

I'm open to any other input, though
http://melka.one.free.fr/blog/
http://www.flickr.com/photos/melkaone/

Syvwlch

This may be a stupid suggestion, but I've had good results taking a leaf from GPS NMEA strings. I've just never tried it quite that fast (more like 9 bytes 5 times a second).

I start a sentence with a special character, say '$', then two-three characters describing the sentence structure if there are several possibilities, then a series of values separated by commas, and stop a sentence with a carriage return. My values uses three bytes and only one sentence structure, but in your case a comma every two or four bytes makes more sense, with perhaps a different sentence header for each line of the matrix?

You can then use the GPS NMEA string reading code from the examples with some modifications. You get the occasional garble, but it resets at every sentence. Your code could stick incoming serial data into an array until it gets a carriage return, recognize the header to ID the line, then split it at the commas, and finally decode the values between the commas. Update matrix, start over with contents of serial buffer.

I dunno if this would work fast enough for you, tho.
----------
Mathieu

melka

One thing that I don't really undertand is that I send 8 bits integers on my line, so they varry from 0 to 255, obviously. Won't the $ character or the carriage return be interpreted as their ASCII equivalent, hence 13 for the carriage return if i'm correct. But 13 will light me leds 1, 3 and 4, no ?

I'll give it a try, right now I managed to do what I wanted (with some weird results from the proMIDI library of processing, but well, at least it's not the Arduino ^^) and it works as a prototype, so for the time being, I'll leave it as is.
But for the final version (something like two 24*24 white leds matrix to control), I'll give it a go, for sure.
http://melka.one.free.fr/blog/
http://www.flickr.com/photos/melkaone/

kg4wsv

Quote
the carriage return be interpreted as their ASCII equivalent, hence 13 for the carriage return if i'm correct.

That one can be tricky, depending on the OS, terminal application, or device you are talking to. Some use a linefeed (ASCII 10), some use CR, some use a CRLF pair.

But in general, yes you are correct.  The data sent down the wire is the binary number representing the ASCII code for that character - that's pretty much what ASCII is all about.

-j


Syvwlch

Depending on how much overhead you have left in your bandwidth, you might want to NOT send your data as bytes but as chars (Three bytes per byte, so not super efficient) or to reserve certain values for control characters (might be difficult for you since you have exactly 8 bits of data per line) or to preface each pair of value bytes with, say, a byte set to zero and interpret each triplet as a three-byte number and anything over 65,535 is garbled data.

You lose bandwidth but you gain legibility both for a human peeking at the exchange and for the arduino.
----------
Mathieu

melka

Maybe I should use the same structure as the monome 40h or series protocol.
http://docs.monome.org/doku.php?id=tech:protocol:40h
http://docs.monome.org/doku.php?id=tech:protocol:serial:series

I already implemented the series serial protocol on another project, and I didn't have any problem at 115200bps.
It takes 2 bytes per row (16 bytes for the whole screen) and works well.
http://melka.one.free.fr/blog/
http://www.flickr.com/photos/melkaone/

Syvwlch

Hey, thanks for sharing those, I didn't know them! :-)
----------
Mathieu

BigMike

The way to send binary data over a serial link is called framing. One uses an "escape" character, DLE (value 0x10) by convention, to "escape" the control elements of the framing.

Usually a frame would take the form of a start byte (STX) and an end byte (ETX) with values 0x02 and 0x03 respectively.

As an example, the following data:

A DLE B ETX DLE STX E

would be framed and sent as:

DLE STX A DLE DLE B ETX DLE DLE STX E DLE ETX

Google DLE STX brings up some relevant documents.

Mike


Grumpy_Mike

So how do you make sure that the STX and ETX are not incidentally in the data you send?

What you need is a unique code which is difficult for 8 bits of data.

One trick you might like to use is to transmit a synchronising start byte as some value say 0x02 BUT transmit it using the wrong parity, then when you get a 0x02 with the wrong parity you know you have a valid start and you can read the next n bytes knowing their sequence.

Failing that your best spiting it up into nibbles and using the top bits to uniquely identify data and header.

kg4wsv

Quote
So how do you make sure that the STX and ETX are not incidentally in the data you send?

By quoting with the DLE.  The real problem is what happens when DLE is in your data; that solution is to "byte stuff" - replace a single DLE with two consecutive DLEs on TX.

-j


BigMike

Mike,

kg4 has it right.  This DLE STX thing is suitable for sending binary data - all 256 combinations - is commonly used and is robust.

I have implemented it several times in the past and included checksums to test for data integrity, and in one application I included code to facilitate data correction.

Regards,

Mike

mikalhart

#12
Sep 15, 2008, 07:20 pm Last Edit: Sep 15, 2008, 07:22 pm by mikalhart Reason: 1
First of all, it's good to see both Big and Grumpy in the same thread. :)

I've been thinking about how the Arduino HardwareSerial class is pretty good for formatted text, but it would be nice to extend it for binary data.  Wouldn't it be useful to include, in addition to the print* suite of functions, something like:

Code: [Select]
void HardwareSerial::writebuf(byte *buffer, int bufsize);
void HardwareSerial::readbuf(byte *buffer, int *bufsize);

// sample usage:
byte buf[64];
Serial.writebuf(buf, sizeof(buf));

int size = sizeof(buf);
Serial.readbuf(buf, &size);


For the readbuf method, you would send the maximum size in and then the actual size would be returned.  Doesn't this seem like it would useful?  I would be happy to contribute to the implementation, along the lines of the discussion above.

Mikal

mem

Quote

First of all, it's good to see both Big and Grumpy in the same thread.

I've been thinking about how the Arduino HardwareSerial class is pretty good for formatted text, but it would be nice to extend it for binary data.  ... Doesn't this seem like it would useful?  I would be happy to contribute to the implementation, along the lines of the discussion above.

Mikal


It's good to see all the above posters here together in one thread  ;)

Fwiw, I haven't seen a big need for methods to send and receive raw binary  packets on the serial port. What I have noticed is a need for an easy way to handle messages that  include parameters of various types. At one point I did write a protocol handler that passed bytes, ints, arrays  and string parameters over the hardware serial port and/or i2c. I may have included the code in a thread a few months ago but felt it needed a lot of work to make it easy enough for general use.

But I would be happy to work with you guys on something along the lines of a simple and extendable protocol handler for arduino/arduino and arduino/pc messaging if there was consensus  that this is something the Arduino community needs and someone could outline the requirements.

melka

Nice to see that this thread took this direction ^^
http://melka.one.free.fr/blog/
http://www.flickr.com/photos/melkaone/

Go Up
 

Quick Reply

With Quick-Reply you can write a post when viewing a topic without loading a new page. You can still use bulletin board code and smileys as you would in a normal post.

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Note: this post will not display until it's been approved by a moderator.
Name:
Email:

shortcuts: alt+s submit/post or alt+p preview