next topic »

[on:]

Hello. I've been dealing quite a lot with serial communication lately. I don't have a computer science degree and lots of questions are crossing my mind. I'll try to be as objective as possible.

Lets take the "File > Examples > Communication > SerialCallResponse" example code and consider the following excerpt:

Code:

   firstSensor = analogRead(0)/4;
    delay(10);
    secondSensor = analogRead(1)/4;
    thirdSensor = map(digitalRead(2), 0, 1, 0, 255);  
    Serial.print(firstSensor, BYTE); // serialPrint number1
    Serial.print(secondSensor, BYTE); // serialPrint number2
    Serial.print(thirdSensor, BYTE);   // serialPrint number3          

1st question: Is it possible that Processing or whatever is connected to Arduino gets the first Serial.print (serialPrint number1) and serialPrint number3, but not serialPrint number2? Or maybe serialPrint number2 and number3 but not the number1?

2nd question: If it could possibly happen, what would be a smart and efficient way to check and correct this kind of misbehavior? Is this what could be called error detection and correction

More questions to come depending on the answers of these two
Thanks a lot in advance,
Luckygoose

[Reply #1 on:]

1. Do you mean like extract number 1 and 3, while you keep the byte 2 in the Serial buffer? Why do you want to do that?

I think, the way Arduino works out the box, the answer is not. But probably hacking the Serial library you can do some trick to achieve that.

2. Again, sometimes is better to explain your final purpose than an unknown possible way to solve.

If you need to identify what you receive, you can use some simple "encoding" like:

0xFF 0x01 0x??  for data from first sensor
0xFF 0x02 0x??  for data from 2nd sensor... and so on

[Reply #2 on:]

1) yes it is possible
2) a checksum could be used

[Reply #3 on:]

The Arduino just sends out the three values, it's up to the receiving application if it can handle the speed of arrival or not. Many possible solutions if you find the PC application is dropping or missing characters, such as:

slow the baud rate down until the PC application doesn't drop characters.

implement a 'handshake' protocol in your Arduino code to only send one byte at a time and wait for a 'response character' from the PC before sending another. Xon/Xoff characters is one common protocol method.

implement hardware control signaling 'hand-shaking' this will require coding and external hardware changes/additons for both ends of the serial link. Often called request to send/clear to send protocol.

Etc, etc, etc.

Try slowing the baud rate first, it's simple and free.  

Lefty

[Reply #4 on:]

To supplement what zoomkat said, yes it is possible. Correcting the dropped bytes is generally not possible. Detecting them often requires changing how you send the data.

If you send a stream of bytes, and one gets dropped, you don't know whether the 1st, 2nd, or 3rd byte of a packet was lost.

If, instead, you send a string, with start-of packet and end-of-packet markers, as well as separators between the numbers, lost data is easily detected.

Suppose you send "<123,456,789>". When reading the string, you can see that there is a start of packet marker (<), and end of packet marker (>), and two separators (,). There is a reasonable expectation, then, that the entire packet was sent correctly.

You can add to the certainty, by sending something like "<3:123,2:45,4:6789>" where the number of characters in each number is also sent. If the number of characters in the number is dropped, you'd have a ,: situation or a <: situation. If a character in the number was dropped (4:689), you'd know that, since the received string length (3) does not match the expected string length (4).

Checksums are another way of determining whether sufficient data was sent. They need to be computed on both ends, and sent and compared.

Whatever you do to determine that ALL the data was received results in the need to send more data. Balancing the amount of data sent versus the likelihood of dropped characters will require some thought.

[Reply #5 on:]

Thanks a lot guys and thank you PaulS for the complete explanation. This is exactly what I was looking for
It is funny to see tons of examples and tutorials around and pretty much none of them talk about that. I think it would be a great addition to the example file that comes with Arduino if it could at least talk a bit about that in the comment area of the code.

1. One further question:
PaulS mentioned sending a string instead of numbers (bytes). Is this the best or easier way to go? It will need more packets (bytes) to send a number, right? One byte per character. Most of the times I will be using Arduino to send and receive numbers only and not text. Should I just stick to send numbers as characters?
I know this is a hard question to answer. It depends a lot on the speed of the connection, the amount of numbers I'm sending, etc, but I just want some thoughts on the matter.

2. Last question:
Is there a complete code example or tutorial that shows an implementation as PaulS is suggesting?

Yesterday I was playing with 2XBee modules and with the SerialCallResponse example. It was actually funny to see lost packets causing the order of things being changed in processing. The pot1 would control the X pos of a circle. The pot2 would control Y and pot3 ( I used an xtra pot) would control the radius. If I took the sending XBee too far away, when it got back into range the entire thing would shift:
pot1 = radius, pot2 = xpos, pot3 = ypos. Quite funny

All the best,
Luckygoose

[Reply #6 on:]

Most of the times I will be using Arduino to send and receive numbers only and not text. Should I just stick to send numbers as characters?
I know this is a hard question to answer. It depends a lot on the speed of the connection, the amount of numbers I'm sending, etc, but I just want some thoughts on the matter.

The real issue is what happens when data is lost. If you are just sending the radius, in your example, and a radius gets dropped, that's no big deal. You just don't draw that circle.

When loosing a value causes the other values to change meaning, it is a much bigger deal. In this case, you need packet markers as I suggested.

Whether to send the numbers as bytes or as strings depends on the application. If the numbers are byte sized (0 to 256), sending as bytes is possible. Text and bytes can be mixed in a packet. When the number is an int, it needs to be sent (and received) as two bytes or as a string.

Sending as a string requires converting the number to a string (using itoa) and the received string back to an int (using atoi). Sending as two bytes requires extracting the most significant byte and the least significant byte, sending them, and reassembling them.

The advantage of sending as a string is that is easier to comprehend the input when echoing it in the Serial Monitor. The drawback is that there may be as many as 6 bytes sent (for a number like -12,345) versus just 2 bytes.

[Reply #7 on:]

The real issue is what happens when data is lost. If you are just sending the radius, in your example, and a radius gets dropped, that's no big deal. You just don't draw that circle.

In my case something worse was happening. The pot that would control the XPos starts to control the YPos or radius. So I definitely need some kind of error detection

The advantage of sending as a string is that is easier to comprehend the input when echoing it in the Serial Monitor.

This is yet another subject. Since I'm experimenting with XBee I've "forced" to use X-CTU under windows. I would be rather using my Mac, but I'll tell you something. X-CTU rocks! It lets you see the hex values for transmitted data, which makes things MUCH easier to debug. Is there such an option or application on OS X that would do that?

Thanks a lot for all your help PaulS. If by any chance you see a code example or tutorial showing an implementation with error detection, please drop me a message.

[Reply #8 on:]

http://www.arduino.cc/playground/Code/Messenger

The scenario for which this library was written is the inverse of yours: the Arduino receives strings and acts appropriately.
However, I think the approach can be used also if processing is receiving strings.
I hope this code gives you some hints.

Also, search the arduino site for something like "serial protocol": a lot of interesting material will show up.

HTH

[Reply #9 on:]

PaulS mentioned sending a string instead of numbers (bytes). Is this the best or easier way to go?

I've worked for some time on implementing a serial protocol between a PIC micro and an Arduino board. I tried both the ascii way and the binary way.

With a binary protocol you have two major problems:
1) byte order
2) packet markers

Problem 1 is solved by carefully writing and debugging (on the wire) the code which serializes and reconstructs multi byte numbers.

Problem 2 requires some thought and some coding. I've found useful hints in the HDLC specs. Basically you need two special bytes: a packet marker and an escape byte.
Whenever you want to express either byte in you packet payload, you have to escape those bytes, thus sending two bytes (escape + marker or escape + escape) over the wire. All the other bytes are sent as they are.
On the receiving end, whenever you encounter an escape byte in your payload, you discard it and take the next byte.
The payload is the stream of bytes between two packet markers.

If you use ascii encoding things are a bit simpler (IMHO).
You take one char as packet marker, or one as start and another as end, and send numbers with itoa or printf.
In my experiments I usually choose * for packet start and # for packet end. Everything in between is the payload. Sometimes I use the two chars right before the end mark as a hex representation of the payload checksum.

HTH