Synchronisation: Reserve byte in serial communication?

Hey, I want to send large integer readings from sensors that are attached to the Arduino. The readings are 16 bit values. I want to send the values in binary form. These will have to be split up into 2 bytes.

Let's consider I have two readings. I am looking for a way to synchronise the serial input, so the computer knows which bytes have to be combined to words.

I have thought of the following approach: If I send it in raw form:

red = reading 1, green = reading 2

... [HiByte] [LoByte] [HiByte] [Lobyte] [HiByte] [LoByte] [HiByte] [Lobyte] ...

The computer might interpret this as:

... [HiByte] [LoByte] [HiByte] [LoByte] [HiByte] [LoByte] [HiByte] [LoByte] ...

Am I right?

Could I reduce the range of the bytes to 0x00 - 0xFE and send an extra byte, let's say 0xFF as a synchronisation marker? The value 0xFF should NEVER be in the encoded readings. I know I will lose some of the range.

... [0xFF] [HiByte (0x00-0xFE)] [LoByte (0x00-0xFE)] [HiByte (0x00-0xFE)] [Lobyte (0x00-0xFE)]
[0xFF] [HiByte (0x00-0xFE)] [LoByte (0x00-0xFE)] [HiByte (0x00-0xFE)] [Lobyte (0x00-0xFE)] ...

Now, the standard code to encode high bytes breaks down:

var myVal = 20000;
var bytes = [];
bytes[0] = (myVal & 0xFF00) >> 8;
bytes[1] = (myVal & 0x00FF);

Does someone know an approach to reserve a single byte value with 16-bit values sent in binary form?

My suggestion is that you reduce the range to 0 to 0xFD and use FE as a start marker and FF as an end marker.

Have a look at the 3rd example in Serial Input Basics. With minor modification it should work with your data.

Another approach is the set aside 3 byte values and use the third one (for example FD) as an indicator that the byte following it is not a marker byte. So, if your data needs to contain FE (for example) you would send FD FE. And if it needs to contain FD you would send FD FD.

However it is much easier to debug a program if the numbers are sent as text and I would only send binary data if the extra performance was actually necessary.

...R

dvtxc:
I have thought of the following approach: If I send it in raw form:

red = reading 1, green = reading 2

... [HiByte] [LoByte] [HiByte] [Lobyte] [HiByte] [LoByte] [HiByte] [Lobyte] ...

The computer might interpret this as:

... [HiByte] [LoByte] [HiByte] [LoByte] [HiByte] [LoByte] [HiByte] [LoByte] ...

Am I right?

Why would there be interleaving of the bytes?

.

ieee488:
Why would there be interleaving of the bytes?

.

Because serial data transmission is NOT guaranteed to happen without corruption now and then.

Robin2:
Have a look at the 3rd example in Serial Input Basics. With minor modification it should work with your data.

I have read your tutorial before I posted this. However, if I understand correctly, you are using ASCII text in that third example, which makes it relatively easy to set escape characters or bytes, doesn't it?

Robin2:
My suggestion is that you reduce the range to 0 to 0xFD and use FE as a start marker and FF as an end marker.

Well, I am stuck at the point on how I should calculate the least significant byte with the reduced range and the carry over to the most significant byte. I guess this should in principle be a simple "base converter", right? Instead of using base 256, you use base 253. My attempts to write a code that does this in c weren't successfull so far..

For now, I have thought of a dirty trick, to just use 7 bits. However, this reduces the range to 0 - 16383. It is not a problem for my current sensors, as they go up to about 8000, but it feels like an easy trick, which is far from efficient, as I am drastically reducing the range.

 // Sync byte. Should be anything over 0x7F
 buff[0] = 0xFF;
 
 // Convert first reading to byte array. [MSB] [LSB]
 buff[1] = (byte) ((reading0 & 0x3F80) >> 7);
 buff[2] = (byte) (reading0 & 0x007F);
 
 // Convert second reading to byte array [MSB] [LSB]
 buff[3] = (byte) ((reading1 & 0x3F80) >> 7);
 buff[4] = (byte) (reading1 & 0x007F);

ieee488:
Why would there be interleaving of the bytes?

.

I have to admit, my understanding of low level computer communication protocols is very bad. Though I don't know why this would not happen. Secondly, I guess this enables the use of a variable amount of sensors, without prior knowledge of the receiving device.

PaulS:
Because serial data transmission is NOT guaranteed to happen without corruption now and then.

Corruption of bytes sent I understand.
Interleaving of bytes where bytes are out of order sent I don't understand.

dvtxc:
I have to admit, my understanding of low level computer communication protocols is very bad. Though I don't know why this would not happen. Secondly, I guess this enables the use of a variable amount of sensors, without prior knowledge of the receiving device.

I have no idea what you are trying to say.

dvtxc:
Well, I am stuck at the point on how I should calculate the least significant byte with the reduced range and the carry over to the most significant byte. I guess this should in principle be a simple "base converter", right? Instead of using base 256, you use base 253. My attempts to write a code that does this in c weren't successfull so far..

I have given you a solution for a system in which all of the 0-255 values can be included in a message and still have start- and end-markers.

I think it is time for you to explain why you need to send binary data rather than convert the numbers to text before sending.

...R