sending and receiving byte arrays (via serial or TCP)

[system]

Hey,

I unfortunately didn't find any satisfying answer to my problem.
I plan to send a byte array from a PC to my aduino either over the serial port or over WLAN via TCP (in the last case equipped with an official Wifi-Shield).
The programming on the PC will be done with VB or C++.

So far, I can sent and receive a string via serial (VB on PC-side). Is there any possibility to directly send and receive a byte array, so that I don't have to use any separating characters and so that I don't have to reassemble the array on my arduino? The examples I found all are reading a string if I am not mistaken.
Is this possible via serial and via a connection over TCP?

Thanks a lot for your help!
squatina

[system]

Is there any possibility to directly send and receive a byte array

That's all you are sending.

So far, I can sent and receive a string via serial (VB on PC-side).

So, lets see that code.

I plan to send a byte array from a PC to my aduino

For what purpose?

[Robin2]

This demo may be close to what you want.

...R

[system]

Sorry for the late reply...

Thanks for the link, that's pretty interesting!

PaulS:

Is there any possibility to directly send and receive a byte array

That's all you are sending.

But how do I handle it then? Like below?

PaulS:

So far, I can sent and receive a string via serial (VB on PC-side).

So, lets see that code.

Here are the important parts of the code:
VB (this code basically sends a string of several bytes, which are stored in a,b,c,...) :

            SerialPort1.Write(x & y & z & a & b & c & d & e & f)

Here's the code on arduino-side:

 if (Serial.available())
{
      Serial.flush();
      delay(5); 
      i=0;      
      while(i<9)
      {
       buffer[i] = serial.read();
                i++; 
       }
      Serial.flush();
 inpos = atoi(X_buffer);
 
... more code
}

Is it also possible to send an array like this?

            SerialPort1.Write(data(9))

How do I receive this on arduino-side?
Like this:

char buffer[9];
int i=0;
while(serial.available() && i<9)
    buffer[i] == serial.read();

Is this possible?

PaulS:

I plan to send a byte array from a PC to my aduino

For what purpose?

To send sensor and encoder data back to the computer. There will be several sensors and encoders and so on, so I thought an array might be best so that the values are related to the specific sensors through their array-index,...

[system]

VB (this code basically sends a string of several bytes, which are stored in a,b,c,...) :

Why are you concatenating stuff like that? Use an array, like a normal person.

 if (Serial.available())
{
      Serial.flush();

Do you have a clue what flush() does? If so, please tell us why, in great detail, you are calling that function here. If not, please tell us why, in great detail, you are calling that function here.

      while(i<9)
      {
       buffer[i] = serial.read();
                i++; 
       }
      Serial.flush();

If there is at least one byte to read, read all 9 of them. Does that make sense to you? It doesn't to me.

You have some dumb idea that serial data transmission is guaranteed. It is not. Get over it.

Is it also possible to send an array like this?

No. data(9) is NOT an array.

Is this possible?

Yes, but that will NOT read 9 bytes.

To send sensor and encoder data back to the computer.

You seem to be confusing the meaning of to and from. If you want to send sensor and encoder data FROM the Arduino, then the Arduino wouldn't be reading anything from the serial port. It would be writing TO the serial port.

Why are you sending an array TO the Arduino?

[system]

Hi, thanks for the quick reply!

PaulS:

VB (this code basically sends a string of several bytes, which are stored in a,b,c,...) :

Why are you concatenating stuff like that? Use an array, like a normal person.

That's exactly what I am planning to do as stated in my first post. You asked me for the code with string-transmission and that's why I posted it...

PaulS:

 if (Serial.available())

{
      Serial.flush();

Do you have a clue what flush() does? If so, please tell us why, in great detail, you are calling that function here. If not, please tell us why, in great detail, you are calling that function here.

Ok, you're right, doesn't make much sense here. Sorry. Waits for the outgoing serial data to complete. Which I don't have here...

PaulS:

      while(i<9)

{
      buffer[i] = serial.read();
                i++;
      }
      Serial.flush();

If there is at least one byte to read, read all 9 of them. Does that make sense to you? It doesn't to me.

You have some dumb idea that serial data transmission is guaranteed. It is not. Get over it.



> Is it also possible to send an array like this?


No. data(9) is NOT an array.

Yes, doesn't make much sense, I need all 9 byte to be sent so that I can read them...
And yes, I know this isn't guaranteed....

And yep, I meant data[] with an array size of 9. Sorry, I was in a bit of hurry, wenn I wrote the answer... So is it possible to send it like this

 SerialPort1.Write(data[]))

PaulS:

Is this possible?

Yes, but that will NOT read 9 bytes.

Why not? Ok, I forgot i++, but otherwise would it do so? And if not, why not. How do I manage to read those 9 bytes?

PaulS:

To send sensor and encoder data back to the computer.

You seem to be confusing the meaning of to and from. If you want to send sensor and encoder data FROM the Arduino, then the Arduino wouldn't be reading anything from the serial port. It would be writing TO the serial port.

Why are you sending an array TO the Arduino?

Sorry, you are right, later on I need both directions. Right now I am concentrating on the part from PC to arduino and somehow I thought you would ask why I also needed the other direction (which I even didn't mention in my first post ). My mistake...
From the PC to the arduino it is to control several servos and motor controllers...
Sorry again!

[system]

I need all 9 byte to be sent so that I can read them...

Then, your Serial.available() test should involve the number 9, not the number 0.

And yep, I meant data[] with an array size of 9. Sorry, I was in a bit of hurry, wenn I wrote the answer... So is it possible to send it like this

No. If you have an array called data, then SerialPort1.Write(data) is the proper way to send it. (Though you may need to tell the function how much data is in data).

Why not? Ok, I forgot i++, but otherwise would it do so? And if not, why not.

Serial data arrives slowly. Suppose that one byte has arrived. Correcting the case and adding the missing increment statement and other errors, your code would look like this:

int i=0;
while(Serial.available() && i<9)
{
    buffer[i] = Serial.read();
    i++;
}

The first part of the conditional expression is true (there IS a byte to be read). The second part is true (0 is less than 9). So, the byte is read and stored, and i is incremented to 1.

Now, there is no more serial data to read, so the first part of the conditional expression is false. The second part is not evaluated. The body is skipped, and the while statements ends, having read just the available data, not 9 bytes.

[system]

PaulS:

And yep, I meant data[] with an array size of 9. Sorry, I was in a bit of hurry, wenn I wrote the answer... So is it possible to send it like this

No. If you have an array called data, then SerialPort1.Write(data) is the proper way to send it. (Though you may need to tell the function how much data is in data).

I see, thank you!

PaulS:
Serial data arrives slowly. Suppose that one byte has arrived. Correcting the case and adding the missing increment statement and other errors, your code would look like this:

int i=0;

while(Serial.available() && i<9)
{
    buffer[i] = Serial.read();
    i++;
}

The first part of the conditional expression is true (there IS a byte to be read). The second part is true (0 is less than 9). So, the byte is read and stored, and i is incremented to 1.

Now, there is no more serial data to read, so the first part of the conditional expression is false. The second part is not evaluated. The body is skipped, and the while statements ends, having read just the available data, not 9 bytes.

So how do I handle this? If I am constantly receiving an array of 9 byte-values every lets say 100ms (if this is enough time for the data transfer), and I always want to refresh buffer[] with those 9 bytes, how do I ensure that there are 9 bytes available (is 'if (Serial.available()==9)' sufficient?)? And is there any way of guaranteeing that I won't start writing with for example byte 5 (so that the bytes written are 5,6,7,8,9,1,2,3,4 which is the wrong order? Is there any way of doing this without any markers? That's also why I thought of arrays but that might get jumbled as well, right?
Is there a way of only having exactly 9 bytes in my serial-buffer and having to erase it before it is allowed so save the next 9 bytes?

Thank you for your patience!

[system]

is 'if (Serial.available()==9)' sufficient?

if(Serial.available() >= 9)
{
}

is.

And is there any way of guaranteeing that I won't start writing with for example byte 5 (so that the bytes written are 5,6,7,8,9,1,2,3,4 which is the wrong order?

No. That's why sending arrays of bytes is so challenging. Arrays of chars are so much easier to parse and assure synchronization and deal with dropped/mangled bytes.

Is there a way of only having exactly 9 bytes in my serial-buffer and having to erase it before it is allowed so save the next 9 bytes?

Nope. If you think about it, the "before it is allowed" bit there is key. What is "it" that is allowed to store data?

[system]

PaulS:

And is there any way of guaranteeing that I won't start writing with for example byte 5 (so that the bytes written are 5,6,7,8,9,1,2,3,4 which is the wrong order?

No. That's why sending arrays of bytes is so challenging. Arrays of chars are so much easier to parse and assure synchronization and deal with dropped/mangled bytes.

Why are char-arrays easier to parse? Why is there a difference? Because of possible starting and ending delimiters when using chars?

So what I basically have to do, is to check for those delimiters and the size of the received array (9 without delimiters) and then only use those arrays?
Which means... I constantly have data arriving, so I fírst look for a starting delimiter, then save the data to an array until I receive an ending delimiter and check the size of the array. Then I start from the beginning again.

And coming back to TCP transmission later on - I don't have to do these checks as this protocol checks the order and the integrity of the data?

[system]

Why are char-arrays easier to parse?

Because parsing tomato soups cans is a real bitch.

So what I basically have to do, is to check for those delimiters and the size of the received array (9 without delimiters) and then only use those arrays?

The size of the array is completely irrelevant. You (should) have a start of packet marker and an end of packet marker. Read and store the data starting when the start of packet marker arrives, and stopping when the end of packet marker arrives. When the end of packet marker arrives, parse the data, converting the tokens back to ints, floats, or tomato soup cans, as appropriate.

and check the size of the array. Then I start from the beginning again.

No, and yes.

And coming back to TCP transmission later on - I don't have to do these checks as this protocol checks the order and the integrity of the data?

No. The TCP layer assures that the TCP packet containing the data got through, unmangled, and in the correct order. It cares not a whit that your data is in any particular order.

[Robin2]

If you always receive 9 bytes then a simple solution is for the Arduino to send a message (perhap 'M' for more) to the PC when it is ready for the next 9 bytes and then use if (Serial.available() >= 9) to know when they arrive. That way you can be sure that you read them in the correct order. To be doubly sure you could empty the buffer with

while (Serial.available() > 0) {
   byte wasteBin = Serial.read()
}

before asking for more data.

If you don't always have the same number of bytes then you need a system of start- and end-markers that allow the Arduino to identify the correct data. This demo illustrates that. At the start if that Thread is a more complex version that can transmit bytes of any value.

...R

[system]

The project that OP is working on is going to eventually send data in the other direction, and that data will not be fixed length. So, learning to collect a packet, and parse it, is going to be necessary on the other end. So, OP might as well learn how to do that now.

OP should NOT assume that a packet is exactly 9 bytes. Serial data does get lost, now and then, after all.

[system]

Thank you both!

PaulS:
The project that OP is working on is going to eventually send data in the other direction, and that data will not be fixed length. So, learning to collect a packet, and parse it, is going to be necessary on the other end. So, OP might as well learn how to do that now.

OP should NOT assume that a packet is exactly 9 bytes. Serial data does get lost, now and then, after all.

And that's exactly why I wanted to check if the array has a specific size... to be sure if data got lost in between the markers. Which now doesn't seem to be very logical, when I think about it as the array will always have the size I gave it in the beginning, just some fields won't get new values in case of data loss.
So I have to check how many values are in between the two markers...