Sequencing data sent through network

Using Arduino Networking, Protocols, and Devices
1

Hi

I am working on a project at the moment. In summary:

  1. I have one device (client) streaming audio via TCP/IP. It samples a 16 bit number (the amplitude) from a sensor.

  2. The 16 bit number is then split into 2 bytes (because TCP/IP can only send in bytes) and sent over TCP/IP. So, for example, the data looks like this:

[Byte 1][Byte 2][Byte 1][Byte 2]...

  1. I have another device (server) which collects and processes the data - it effectively joins the bytes back up to get a 16 bit number.

The problem is ensuring that the server knows which bytes are in pairs (otherwise it may join the wrong bytes). The server may start listening after the client sends its first data so we cannot rely upon the first byte received as being the first byte in a pair.

I initially tried to solve this through TCP packets - effectively the client sends a packet of bytes where the first byte in the packet is guaranteed to be the first in a byte pair. But this doesn't work as it seems that TCP/IP libraries don't expose specific packets but only a stream of bytes.

Another solution is to use a delimiter between a large group of bytes (and then just wait until a delimiter is received so we know the next byte is byte 1 in a pair). I guess that means I'd have to set a byte aside with a value of somewhere in 0 - 255 to act as a delimiter? The problem with this approach, as I see it, is that one byte in a pair might happen to be the delimiter.

I'm finding this surprisingly difficult to solve so grateful for any thoughts. I'd like to keep it as simple as possible so ideally it'd all be one way (client to server) rather than sending acknowledgements back etc.

Thanks

2

UDP out of the question?

3

yes, maybe but i'm not sure how that helps solve the problem?

4

A UDP packet will contain only those 4 bytes in the order in which you placed them.

5

This is not a new problem, and some method of synchronizing the data is required. Most people use packets of some sort, often with data start and end markers.

If the data are continuously streamed, then a longer, unique preamble to identify a start sequence may be required.

6

thanks, can you link me to any good resources on this?

7

Look in the IDE examples.
Ethernet -> UDPSendReceiveString
Don't be fooled by the title. It's a UDP "client" sketch. It actually receives and replies to a UDP "server".
Use the same in reverse to be the "server".

8

Are there any libraries which I can use for this? Currently using wifi 101. thanks

9

Here is an example for wifi 101. Basically same but for wifi. Same protocol, different transport.

10

A simpler approach may or may not be relevant.
If your sensor's actual return value is 14 bits or less, simply put 7 bits in each byte, setting the eighth bit of the first byte, and clearing the eighth bit of the second byte. In that way, you always know what's what.
Just another option. I've used this method to allow streaming of 12-bit data without adding overheads.

1 Like
11

Pretty simple..
Need to design a simple protocol..
Each packet should start with a simple header..
I tend to use packed structures..

Example tcp client..

good luck.. ~q