Designing a Serial Protocol.

Good day,

I have a wif to serial converter which i can plug my ardruino into its usb and it then connects to the wifi, I then create a virtual com port on my pc and i can then talk serial to the computer.

I plan to use this for a mobile robot / rc car. What i want to enquirer about is that i want to design a reliable and efficient serial protocol to talk between my ardruino and my java program that i will be using to control the device.

I want the device to upload a whole bunch of sensor readings every say ~100 ms and then i also want to be able to constantly send it movement commands, ie forward left right etc.
There will also be some commands which which will not be real time. for example turn on camera. this almost needs to implement a tcp sort of packet where it insures / tries it best to get the packet across vs, where if a sensor data reading goes missing here and there its no biggy, so this can implement a UDP type design.

Where can i find some good resources on designing a proper and reliable protocol.

Maybe there is already a library out there that does this that i can use? I mean i doubt i am the first person to do this.

Also, What would be a better design scheme if i do design my own one?
A: to make the arduino act purely as an IO device. And the java app will do all the processing and controll it will jsut tell the ard to turn pin on, read pin etc?

B: hybrid combo where the pc will tell the arduino to move forward and it will then do the necessary commands to move the device forward?

To me option A will be more generic and will be easier as you wont have to access the ardruino every time you make a small change.

What about VirtualWire, I don't know much about it but I'd have a look.


Rob

Could I not make use of Firmata to do this?

Air wire seems to be specifically designed for RF transmission.

What i want to enquirer about is that i want to design a reliable and efficient serial protocol to talk between my ardruino and my java program that i will be using to control the device.

this almost needs to implement a tcp sort of packet where it insures / tries it best to get the packet across vs, where if a sensor data reading goes missing here and there its no biggy, so this can implement a UDP type design.

These two statements don’t go together. Serial data is sent in a stream. It is not packeted and delivered.

PaulS:
These two statements don't go together. Serial data is sent in a stream. It is not packeted and delivered.

It is if you make it. At the end of the day ethernet is also just a wire with a stream of signals.

If you design your protocol to have a "packet structure" you can build a header and then have a body , add a crc at the end and all the things necessary to get your data across.
And with regards to delivered, that is purely how you send and receive your data. if your crc doesnt add up on the receiver side it can tell the sender to send the packet again. (if need be of course).

How good is firmata?

with regards to efficiency over the serial port,

and accuracy as in error checking.
Does it do a crc on the data that it sends / receives?

How good is firmata?

Feel free to look at the source code to answer all of your questions.

I cant say i find that post very helpful to be honest. If i wanted to JUST look at the code dont you think i would have done that already?

I was asking to get peoples personal experiences with using it. If people have had lots of trouble with it then i know not to waist my time. If it has worked wanders for everyone then i know its worth a bash.

So if you have any constructive opinions please feel free to comment.

If people have had lots of trouble with it then i know not to waist my time.

It isn't that difficult to count the problems with Firmata threads. It isn't that difficult to go to the Exhibition section and count the success stories involving Firmata.

Firmata is available for the Arduino. But, it is not supported. Try to find a single thread where someone says "Hey, yeah, I'm working on fixing that" from someone supporting Firmata. Or, one that says "Oh, yeah, that was fixed in version xxx". Or, ANY thread where anyone from the Firmata camp is supporting the use of Firmata on the Arduino.

But, don't let me stop you from using it. Build a robot with a Ping sensor. Get data from the sensor on the PC using Firmata. Let us know how that goes.

So you are saying you wouldnt go any where near firmarta?

I came across this
https://code.google.com/p/arduino-serial-packet/

It looks like something that I want, It just looks abit unfinished or unpolished. Think Im gonna give it a try.

Hey,

I found your post/question very interesting, specially about the 'writing a serial protocol :slight_smile:

Could you inform me/us on how this project has progressed?

Zapnologica:
What i want to enquirer about is that i want to design a reliable and efficient serial protocol to talk between my ardruino and my java program that i will be using to control the device.

Have a look at the 3rd example in Serial Input Basics

Unless you specifically need WiFi I suspect it would be simpler to use nRF24L01+ 2.4GHz transceivers - they are cheap and they have a huge amount of error checking built in.

...R

You may also wish to read this thread:

Sometimes, using a send-receive structure is a more efficient method of moving formatted-repeatitive data provided you manage your CRC process properly.

Ray

Any time you use serial, you have to expect that the data will get corrupted. Even the fridge switching on in the kitchen will send out a burst of interference that will flip a 1 or 0 to the opposite. With a highly packed data format, that might be the difference between forwards and backwards and the car might damage itself or a person.

You have specified WiFi, which will work flawlessly in the face of severe interference, but you still have the commands "exposed" on a short serial cable. This cannot be trusted to always output a faithful copy of the input.

So the commands going to the car need to include extra bytes that will let it determine if the command is valid and not corrupted. The simplest method is a checksum. Add up all the values of all the bytes in the command and send that sum as part of the message.

Look at the NMEA protocol used by GPS. That is pretty much the simplest way to do a checksum and keep it human-readable for debugging purposes. I use that a lot for command structures.

But a checksum can only tell you that there was an error. It can't tell you where the error is or how to fix it. Often that is enough. If the car doesn't move, you just press 'forward' again.

The next step up is CRC - cyclic redundancy check. Basically a bigger checksum that includes enough extra data that the receiver can fix simple errors and identify complex multi bit errors.

MorganS:
<…>
Look at the NMEA protocol used by GPS. That is pretty much the simplest way to do a checksum and keep it human-readable for debugging purposes. I use that a lot for command structures.
<…>

Yes.

… and a comma delimited protocol is very easy to parse. Adafruit’s GPS library uses a simple technique that is easy to duplicate for most situations:

 if (strstr(nmea, "$GPRMC")) {
   // found RMC
    char *p = nmea;

    // get time
    p = strchr(p, ',')+1;
    float timef = atof(p);
    uint32_t time = timef;
    hour = time / 10000;
    minute = (time % 10000) / 100;
    seconds = (time % 100);

    milliseconds = fmod(timef, 1.0) * 1000;
    p = strchr(p, ',')+1;  // A/V?
    p = strchr(p, ',')+1;  // lat
    p = strchr(p, ',')+1;  // N/S?
    p = strchr(p, ',')+1;  // lon
    p = strchr(p, ',')+1;  // E/W?
    p = strchr(p, ',')+1;  // speed
    p = strchr(p, ',')+1;  // angle

    p = strchr(p, ',')+1;
    uint32_t fulldate = atof(p);
    day = fulldate / 10000;
    month = (fulldate % 10000) / 100;
    year = (fulldate % 100);
    // we dont parse the remaining, yet!
    return true;
  }

Ray

I am going to bump this post, trying to find out the status of your project: "Designing a Serial Protocol."

Also as reminder of the question:

hewi:
Hey,

I found your post/question very interesting, specially about the 'writing a serial protocol :slight_smile:

Could you inform me/us on how this project has progressed?

and as a link to my own new topic: Arduino serial communication: introducing Yaspie