Reading serial and sending via ethernet problems

Hey guys, I'm having some trouble with the following:
Here's what I'm doing: I'm reading the serial port for incoming data, parsing the data (which is sent in a specific format: , for example <1-1023>), and then sending the id and the data to my web server.

I have no problem with the reading the serial port, parsing the data or sending data to the web server, but when I combine all of them I get "corrupted" data from time to time... for example:
my data ranges from 0 to 1023, and I got a couple of times something like 8830, which is over 1023. I am also getting, from time to time, a device ID that doesn't match the current device ID (which is 1).

I'm thinking the delays from sending to the web server affects the serial buffer somehow and messes up my data from time to time? Let me know if you find anything that could fix my problem :slight_smile:

Here's my code:

#include <Ethernet.h>
#include <SPI.h>

#define SOP '<' // Start of packet
#define EOP '>' // End of packet
#define PAD '-' // Packet (device) address delimiter
// Example of a packet: <1-1022> ---> device address = 1, data = 1022

// Ethernet config
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0F, 0x2A, 0x8D };
byte ip[] = { 192, 168, 2, 77 };
byte gw[] = {192, 168, 2, 1};
byte subnet[] = { 255, 255, 255, 0 };

EthernetClient client;//(server, 80);

byte server[] = { 192, 168, 2, 83  }; // Server IP

// Data config -- information to be sent to the web server
int device = 0;
int data = 0;

int index = 0;
char tmp = 0; //temporary variable to store serial data input, that will be parsed.
char in_buffer[8];
boolean end_of_packet = false;

void setup()
{
  Serial.begin(9600);               // Used for serial debugging
  Ethernet.begin(mac, ip, gw, gw, subnet);
  Serial.println("Program running...");
  delay(1000);                                    //Keeps the connection from freezing
}

void loop()
{
  while (Serial.available() > 0) {
    tmp = Serial.read();
    if (tmp == SOP) {
      index = 0;
    }
    else if (tmp == EOP) {
      index++;
      in_buffer[index] = '\0';
      data = atoi(in_buffer);
      end_of_packet = true;
      break; //exit while loop
    }
    else {
      in_buffer[index] = tmp;
      index++;
      if (tmp == PAD) {
        index++;
        in_buffer[index] = '\0';
        device = atoi(in_buffer);
        index = 0;
        memset(in_buffer, 0, 8);
      }
    }
  }
  if (end_of_packet) {
    index = 0;
    memset(in_buffer, 0, 8);
    if ((10 > device >= 1) && (1023 >= data >= 0)) { //if condition was added specifically to prevent some of the corrupted data
      Serial.println("Device ID: ");
      Serial.println(device);
      Serial.println("Data: ");
      Serial.println(data);
      //delay(5000);
      senddata();       // Data is sent
      end_of_packet = false;
    }
    else {
      end_of_packet = false;
    }
  }
}

void senddata()
{

  Serial.println();
  Serial.println("ATE :)");

  if (client.connect(server, 80)) {
    Serial.println("Connected");
    client.print("GET /insertdata.php?");
    client.print("device=");
    client.print(device);
    client.print("&data=");
    client.print(data);
    client.println(" HTTP/1.1");
    client.println("Host: 192.168.2.83");
    client.println("Connection: close");
    client.println();
    Serial.println();
    while (client.connected()) {
      while (client.available()) {
        Serial.write(client.read());
      }
    }
  }
  else
  {
    Serial.println("Connection unsuccessful");
  }
  //stop client
  client.stop();
  while (client.status() != 0)
  {
    delay(5);
  }
}

As you can see, I added a condition to send data so some of the corrupted data isn't sent, but I still get a device ID of 0 from time to time when it's supposed to be 1.

Thanks a lot for your help!

  Serial.println("Program running...");

You are not using the F() macro, to keep string literals out of SRAM. Why not? How much free memory DO you have?

http://playground.arduino.cc/Code/AvailableMemory

http://playground.arduino.cc/Learning/Memory

Ahh I see, thanks a lot for the info Paul.
I really don't use arduino that often, didn't know this macro existed. From now on, I'll use the F() macro in order to save some ram by using the flash instead, seems much more efficient :slight_smile:

I'll fix this later today when I get home.

Edit: Turns out I won't have time today, I'll do it tomorrow ^^
Do you think this could help towards solving the problem I'm currently having?

Alright so I fixed what Paul mentioned, and it seems to have fixed my problem? I'm not sure because I have only been testing the program for a bit, and haven't had any messed up data sent. But, as I mentioned before, it only happened from time to time... I guess only time will tell, so I'll keep it running to see if I get anything :slight_smile:

Here's what the code looks like:

#include <Ethernet.h>
#include <SPI.h>

#define SOP '<' // Start of packet
#define EOP '>' // End of packet
#define PAD '-' // Packet (device) address delimiter
// Example of a packet: <1-1022> ---> device address = 1, data = 1022

// Ethernet config
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0F, 0x2A, 0x8D };
byte ip[] = { 192, 168, 2, 77 };
byte gw[] = {192, 168, 2, 1};
byte subnet[] = { 255, 255, 255, 0 };

EthernetClient client;//(server, 80);

byte server[] = { 192, 168, 2, 83  }; // Server IP

// Data config -- information to be sent to the web server
int device = 0;
int data = 0;

int index = 0;
char tmp = 0; //temporary variable to store serial data input, that will be parsed.
char in_buffer[8];
boolean end_of_packet = false;

void setup()
{
  Serial.begin(9600);               // Used for serial debugging
  Ethernet.begin(mac, ip, gw, gw, subnet);
  Serial.println(F("Program running..."));
  delay(1000);                                    //Keeps the connection from freezing
}

void loop()
{
  while (Serial.available() > 0) {
    tmp = Serial.read();
    if (tmp == SOP) {
      index = 0;
    }
    else if (tmp == EOP) {
      index++;
      in_buffer[index] = '\0';
      data = atoi(in_buffer);
      end_of_packet = true;
      break; //exit while loop
    }
    else {
      in_buffer[index] = tmp;
      index++;
      if (tmp == PAD) {
        index++;
        in_buffer[index] = '\0';
        device = atoi(in_buffer);
        index = 0;
        memset(in_buffer, 0, 8);
      }
    }
  }
  if (end_of_packet) {
    index = 0;
    memset(in_buffer, 0, 8);
    if (device >= 1 && device < 10 && data >= 0 && data <= 1023) {
      Serial.println("Device ID: ");
      Serial.println(device);
      Serial.println("Data: ");
      Serial.println(data);
      //delay(5000);
      senddata();       // Data is sent
      end_of_packet = false;
    }
    else {
      end_of_packet = false;
    }
  }
}

void senddata()
{

  Serial.println();
  Serial.println(F("ATE :)"));

  if (client.connect(server, 80)) {
    Serial.println("Connected");
    client.print("GET /insertdata.php?");
    client.print("device=");
    client.print(device);
    client.print("&data=");
    client.print(data);
    client.println(" HTTP/1.1");
    client.println("Host: 192.168.2.83");
    client.println("Connection: close");
    client.println();
    Serial.println();
    while (client.connected()) {
      while (client.available()) {
        Serial.write(client.read());
      }
    }
  }
  else
  {
    Serial.println(F("Connection unsuccessful"));
  }
  //stop client
  client.stop();
  while (client.status() != 0)
  {
    delay(5);
  }
}

Is this just a happy coincidence, or did clearing up the RAM of useless stuff really helped out a bunch for my problem?

I'll keep this updated and let you know if I get anything wrong or if everything seems to be fine :slight_smile:

Quick update, I am still getting some minor problems: I get some messed up data from time to time (for example, value should be around 504, and I got 0, 4, 5 a couple of times).

Terminal to show you the "messed up" data I'm getting:

Device ID: 
1
Data: 
0

ATE :)
Connected

HTTP/1.1 200 OK
X-Powered-By: PHP/5.4.36-0+deb7u3
Content-type: text/html
Connection: close
Transfer-Encoding: chunked
Date: Thu, 15 May 2014 03:14:41 GMT
Server: lighttpd/1.4.31

42
<!DOCTYPE html>
<html>

Connected successfully 
 

</html>
0

Device ID: 
1
Data: 
504

ATE :)
Connected

HTTP/1.1 200 OK
X-Powered-By: PHP/5.4.36-0+deb7u3
Content-type: text/html
Connection: close
Transfer-Encoding: chunked
Date: Thu, 15 May 2014 03:14:42 GMT
Server: lighttpd/1.4.31

42
<!DOCTYPE html>
<html>

Connected successfully 
 

</html>
0
... and a little later on:
Device ID: 
1
Data: 
4

ATE :)
Connected

HTTP/1.1 200 OK
X-Powered-By: PHP/5.4.36-0+deb7u3
Content-type: text/html
Connection: close
Transfer-Encoding: chunked
Date: Thu, 15 May 2014 03:14:50 GMT
Server: lighttpd/1.4.31

42
<!DOCTYPE html>
<html>

Connected successfully 
 

</html>
0

Device ID: 
1
Data: 
504

ATE :)
Connected

HTTP/1.1 200 OK
X-Powered-By: PHP/5.4.36-0+deb7u3
Content-type: text/html
Connection: close
Transfer-Encoding: chunked
Date: Thu, 15 May 2014 03:14:50 GMT
Server: lighttpd/1.4.31

42
<!DOCTYPE html>
<html>

Connected successfully 
 

</html>
0

Keep in mind, I am not getting these very frequently. I let it run for about 5-7minutes, which ended up sending ~205 different things, and got some messed up data 20 times, so that's not too bad.
Of course, I would love to eliminate the problem, but if not we could find a workaround?
Perhaps a condition that compares the previous value, if it's way too different than don't send it?
I don't need to send it every second, just about every 5 second, so if I don't send one out of many that are currently sent during 5 seconds it doesn't matter :slight_smile:

Thank you for your help!

You seem to be reading and parsing the data at the same time. Have a look at the examples in serial input basics - the 3rd one is probably most relevant. They read all the data first and only parse it subsequently. It has the advantage that you could print the raw received data for debugging. The examples assume a comma as a delimiter. You could change it to a hyphen but I would prefer not (if I had the choice) as it cold get confused with a minus sign.

You seem only to have added the F macro on two lines. But you have many more lines with fixed text -why not use it for all of them?

...R

Thanks for the reply Robin, I'll look into it :slight_smile: My first intention was to do exactly what you just said (parsing it subsequently), but I had some problems doing it, so I'm sure your links will help me greatly.

And regarding the F macro, I noticed a couple of lines where I missed the fixed text after posting my last reply, and have modified the lines consequently ^^

Edit: can't quite seem to get this to work when using strtok, I'll work on it some more tomorrow.

Hi brother6,

you have one pitfall through your whole program. Whenever you are using index++; check for the maximum buffer length. Like this:

if ( index < 8 ) { 
  index++;
}

Otherwise it's possible to have a buffer overflow with all the strange behavior after that.

Greetings,
Nighti

The below is some older test code modified to capture characters sent from the serial monitor, evaluate what was captured, and then connects to a server on the internet. Not exactly what is needed, but has the major components.

//zoomkat 11-04-13
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test client GET
//for use with W5100 based ethernet shields
//remove SD card if inserted
//data from myIP server captured in readString 

#include <SPI.h>
#include <Ethernet.h>
String serialString, serverString;

byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address

char serverName[] = "checkip.dyndns.com"; // myIP server test web page server
EthernetClient client;

//////////////////////

void setup(){

  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }

  Serial.begin(9600); 
  Serial.println("client readString test 11/04/13"); // so I can keep track of what is loaded
  Serial.println("Send \"getip\" in serial monitor to test"); // what to do to test
  Serial.println();
}

void loop(){
  // check for serial input
  while (Serial.available()) {
    delay(3);  
    char c = Serial.read();
    serialString += c; 
  }

  if (serialString.length() >0) {
    Serial.println(serialString);

    if (serialString == "getip")     
    {
      sendGET(); // call sendGET function below when byte is an e
    }
    serialString="";
  }  
} 

//////////////////////////

void sendGET() //client function to send/receive GET request data.
{
  if (client.connect(serverName, 80)) {  //starts client connection, checks for connection
    Serial.println("connected");
    client.println("GET / HTTP/1.1"); //download text
    client.println("Host: checkip.dyndns.com");
    client.println("Connection: close");  //close 1.1 persistent connection  
    client.println(); //end of get request
  } 
  else {
    Serial.println("connection failed"); //error message if no client connect
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    serverString += c; //places captured byte in readString
  }

  //Serial.println();
  client.stop(); //stop client
  Serial.println("client disconnected.");
  Serial.println("Data from server captured in serverString:");
  Serial.println();
  Serial.print(serverString); //prints readString to serial monitor 
  Serial.println();  
  Serial.println();
  Serial.println("End of serverString");
  Serial.println("==================");
  Serial.println();
  serverString=""; //clear readString variable

}

brother6:
Edit: can't quite seem to get this to work when using strtok, I'll work on it some more tomorrow.

If you still have problems post your latest code.

...R

Thank you all for your answers! :slight_smile:

@Robin, here's what my code looks like. I commented out the serial reading, just to see what the problem was, and it seems to be the parsing. I can't separate the data properly.

Here's my code, I'm trying to use strtok but I'm not having much success...

#define SOP '<' // Start of packet
#define EOP '>' // End of packet
#define PAD '-' // Packet (device) address delimiter
// Example of a packet: <1-1022> ---> device address = 1, data = 1022

// Data config -- information to be sent to the web server
int device = 0;
int data = 0;

int index = 0;
char tmp = 0; //temporary variable to store serial data input, that will be parsed.
//char in_buffer[16];
char in_buffer[] = "<1-955>";
boolean end_of_packet = false;

void setup()
{
  Serial.begin(9600);               // Used for serial debugging
  Serial.println(F("Program running..."));
}

void loop()
{
  //storeSerial();
  end_of_packet = true;
  parseData();
}

void storeSerial()
{
  while (Serial.available() > 0 && end_of_packet == false) {
    tmp = Serial.read();
    if (tmp != EOP) {
      in_buffer[index] = tmp;
      index++;
    }
    else {
      in_buffer[index] = '>';
      index++;
      in_buffer[index] = '\0';
      index = 0;
      end_of_packet = true;
      Serial.println(F("Buffer: "));
      Serial.println(in_buffer);
    }
  }
}

void parseData()
{
  if (end_of_packet) {
    char * strtokIndx; // this is used by strtok() as an index
    strtokIndx = strtok(in_buffer, "<");

    strtokIndx = strtok(NULL, "-");
    device = atoi(strtokIndx);

    strtokIndx = strtok(NULL, ">");
    data = atoi(strtokIndx);
    Serial.println(F("Device ID: "));
    Serial.println(device);
    Serial.println(F("Data: "));
    Serial.println(data);
    end_of_packet = false;
  }
}

I still don't quite understand why my original code doesn't work perfectly... without the portion that sends the data to the web my parsing works perfectly fine :confused:
If you guys have a way to just compare the data and not send it if it's messed up let me know, that works too as I don't need to send it as often as I am sending it right now :slight_smile:

If you were using my serial input examples the <> would not be saved in the received data. Try this
char in_buffer[] = "1-955";

Then all you have to parse is two pieces separated by -

If you use my receive code with my parse function you should not have a problem.

...R

I'm currently using the examples Robin previously linked and modifying them to match my situation, so far I'm not having any problems with extracting the received packets.

I'll add the parsing and then will merge it with my sendData function which sends to my web server and let you guys know how it's going :slight_smile:

For the benefit of other readers, the code in Reply #10 is not from my examples.

...R

Yeah sorry, previous code I posted wasn't from Robin's examples. :slight_smile:

I'm still having some problems when using your examples though Robin. When I'm only extracting my serial buffer and displaying it, I don't have any problem. My input varies from 0 to 1023, and what I'm extracting as well.

BUT, the problem comes in when parsing the extracted buffer.

Here's my code, which is basically Robin's example but with some variable names altered, and I added a condition to try to fix my problem (which sadly didn't fix it):

#define SOP '<' // Start of packet
#define EOP '>' // End of packet
#define PAD '-' // Packet (device) address delimiter

const byte buff_size = 16;
char in_buffer[buff_size];
char tmp_buffer[32];        // temporary array for use by strtok() function

int device = 0;
int data = 0;

boolean newData = false;
boolean packetComplete = false;
boolean parsingComplete = false;

void setup() {
  Serial.begin(9600);
  Serial.println(F("<Arduino is ready>"));
}

void loop() {
  extract_serial_packet();
  showNewData();
  strcpy(tmp_buffer, in_buffer);
  //memset(in_buffer, 0, buff_size);
  parseData();
  showParsedData();
  //delay(2000);
}

void extract_serial_packet() {
  static boolean recvInProgress = false;
  static byte index = 0;
  char tmp;

  while (Serial.available() > 0 && newData == false) {
    tmp = Serial.read();

    if (recvInProgress == true) {
      if (tmp != EOP) {
        in_buffer[index] = tmp;
        index++;
        if (index >= buff_size) {
          index = buff_size - 1;
        }
      }
      else if (tmp == SOP) { //I added this to try and fix things, but it didn't ^^
        index = 0;
      }
      else {
        in_buffer[index] = '\0'; // terminate the string
        recvInProgress = false;
        index = 0;
        newData = true;
      }
    }

    else if (tmp == SOP) {
      recvInProgress = true;
    }
  }
}

void showNewData() {
  if (newData == true) {
    Serial.print(F("This just in ... "));
    Serial.println(in_buffer);
    newData = false;
    packetComplete = true;
  }
}

void parseData() {
  if (packetComplete) {

    // split the data into its parts

    char * strtokIndx; // this is used by strtok() as an index

    strtokIndx = strtok(tmp_buffer, "-");     // get the first part - the string
    device = atoi(strtokIndx); // convert this part to an integer

    strtokIndx = strtok(NULL, "-"); // this continues where the previous call left off
    data = atoi(strtokIndx);     // convert this part to an integer
    packetComplete = false;
    parsingComplete = true;
  }
}


void showParsedData() {
  if (parsingComplete) {
    Serial.print(F("Device id: "));
    Serial.println(device);
    Serial.print(F("Data: "));
    Serial.println(data);
    parsingComplete = false;
  }
}

And here's what I'm getting on the serial monitor:

<Arduino is ready>
This just in ... 1-1023
Device id: 1
Data: 1023
This just in ... 1-1023
Device id: 1
Data: 1023
This just in ... 1-1023
Device id: 1
Data: 1023
This just in ... 1-1023
Device id: 1
Data: 1023
This just in ... 1-1023<1-1023
Device id: 1
Data: 1023
This just in ... 1-102<1-1023
Device id: 1
Data: 102
This just in ... 1-1<1-1023
Device id: 1
Data: 1
This just in ... 1<1-1012
Device id: 1
Data: 1012
This just in ... 1-956
Device id: 1
Data: 956
This just in ... 1-904
Device id: 1
Data: 904
This just in ... 1-866
Device id: 1
Data: 866
This just in ... 1-822
Device id: 1
Data: 822
This just in ... 1-755
Device id: 1
Data: 755
This just in ... 1-689
Device id: 1
Data: 689
This just in ... 1-627
Device id: 1
Data: 627
This just in ... 1-569
Device id: 1
Data: 569
This just in ... 1-513
Device id: 1
Data: 513
This just in ... 1-451
Device id: 1
Data: 451
This just in ... 1-400
Device id: 1
Data: 400
This just in ... 1-354
Device id: 1
Data: 354
This just in ... 1-319<1-280
Device id: 1
Data: 319
This just in ... 1-236
Device id: 1
Data: 236
This just in ... 1-143
Device id: 1
Data: 143
This just in ... 1-114
Device id: 1
Data: 114
This just in ... 1-50
Device id: 1
Data: 50
This just in ... 7
Device id: 7
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1<1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-0
Device id: 1
Data: 0
This just in ... 1-<1-0<1-0
Device id: 1
Data: 0
This just in ... 1-40<1-83
Device id: 1
Data: 40
This just in ... <1-200<1-252<1-
Device id: 0
Data: 200
This just in ... <1-537
Device id: 0
Data: 537
This just in ... 1-563
Device id: 1
Data: 563
This just in ... 1-582
Device id: 1
Data: 582
This just in ... 1-597
Device id: 1
Data: 597
This just in ... 1-614
Device id: 1
Data: 614
This just in ... 1-632
Device id: 1
Data: 632
This just in ... 1-652
Device id: 1
Data: 652
This just in ... 1-668
Device id: 1
Data: 668
This just in ... 1-686
Device id: 1
Data: 686
This just in ... 1-705
Device id: 1
Data: 705
This just in ... 1-725
Device id: 1
Data: 725
This just in ... 1-744
Device id: 1
Data: 744
This just in ... 1-757
Device id: 1
Data: 757
This just in ... 1-777<1-797
Device id: 1
Data: 777
This just in ... 1-820
Device id: 1
Data: 820
This just in ... 1-878
Device id: 1
Data: 878
This just in ... 1-904
Device id: 1
Data: 904
This just in ... 1-964<1-994
Device id: 1
Data: 964
This just in ... 1<1-1023
Device id: 1
Data: 1023
This just in ... 1-1022
Device id: 1
Data: 1022
This just in ... 1-1<1-1023
Device id: 1
Data: 1
This just in ... 1-1022
Device id: 1
Data: 1022
This just in ... 1-1023
Device id: 1
Data: 1023
This just in ... 1-1023
Device id: 1
Data: 1023

My input is, to make it short, a potentiometer which varies between 0-1023 (0-5V through and ADC) and the result is sent to the serial port along with the device's id.

Thank you for your help, really want to see how I could fix this :smiley:

    Serial.print(F("This just in ... "));
    Serial.println(in_buffer);

When printing strings, I like to make certain that there is nothing after the string that I can't see.

Serial.print(">");
Serial.print(in_buffer);
Serial.println("<");

Or some other characters that I know are not in the string.

You are making an assumption that serial data is ALWAYS successfully transmitted. As you are seeing, this is not a valid assumption.

For truly reliable serial transmission, you would send the length of the packet, the packet, and a checksum based on the packet contents, as well as the start and end markers. Upon receipt of the start of packet marker, you'd read the expected length. Then, you'd read the data. Then, you'd read the checksum. You'd compare the length of the packet to what was expected. You'd compute the checksum, and compare it with the expected value. You'd discard the packet if the length or checksum didn't match.

Thanks for your input Paul.

The serial information I am trying to retrieve with this program is actually an analog value (0-5V), which is converted to digital with a microcontroller's ADC module, which sends it through serial to an xbee. On the other end, I have the xbee that receives the data and that transmits it to the arduino via the serial port.

I know the length of the packet I need to extract, but its length is actually variable. As you can see in my previous posts, the packet format is the following: <1-1023>, where 1 is the device id, and 1023 is the data. BUT, the data isn't fix, it can vary between 0 and 1023, therefore the packet size can vary as well, which is where my problem seems to be.

I'm currently trying to figure out how to read/extract the information received, but any input on the matter is greatly appreciated! :slight_smile:

I know the length of the packet I need to extract, but its length is actually variable. As you can see in my previous posts, the packet format is the following: <1-1023>, where 1 is the device id, and 1023 is the data. BUT, the data isn't fix, it can vary between 0 and 1023, therefore the packet size can vary as well, which is where my problem seems to be.

Actually, the length IS fixed. The device ID is one byte, and the data is two bytes. Sending binary data means sending less data, AND its generally far easier to "parse" as there really is no parsing involved.

Just a thought.

PaulS:
Actually, the length IS fixed. The device ID is one byte, and the data is two bytes. Sending binary data means sending less data, AND its generally far easier to "parse" as there really is no parsing involved.

Just a thought.

I think I need to sleep a bit... I didn't think about that bolded part -.-

As for binary data, I'm not sure I get what you're trying to say... are you suggesting that I send my data in binary format?

are you suggesting that I send my data in binary format?

If you do, there is:

  1. Less data to send
  2. No parsing required.

So, there are advantages. There are disadvantages, though, that need to be considered.

First, it is much harder to synchronize data. Start and end of packet markers make synchronizing ASCII data easy. Since any start or end marker might be the same as a valid byte you are trying to send, syncing binary data is harder.

Typically, what is done is, if you are sending three bytes in the packet, send three identical bytes, then the data, then the same three bytes. If you don't read three bytes of sync data, three bytes of maybe good data, and three bytes of sync data, then the packet is bad.

Of course, now you are sending nine bytes of data, when only three are useful (except for data integrity purposes).

You can cut that down to 6 bytes, with care. Three sync bytes, then three data bytes.

If you know, for instance that the ID will never be more than some value, then a couple of sync bytes, a value between 0 and n, and two more byte, then 2 more sync bytes mean that the packet is good, and you've got the start of the next one.