Send and recieve data using serial connection (PC)

Hello,

I am making a small program ( VB .NET) that reads my USB GamePad input and sends user defined messages (based on what button is pressed) from the PC to the Arduino.

The Arduino can also send messages back to the PC.

Now I have a little problem (or is it a challenge ;))?

I have made a small program that sends data from the Arduino to my PC. The message "Test" is send every second to my PC. I show the data in a TextBox. I would like to place every message on a new line.

the message is send using the following command: Serial.print("Test")

So:
Test
Test
Test

And not:
Test Test Test

The problem is I get this:

T
est
Test
Te
st

It looks like the message is divided in small packets? is this the normal way? I don't know how data is send over a serial connection?

Is there a common way to send data to a PC and back? Maybe put a special piece of code behind every message, like: Test<$> . So the buffer only reads to <$> ?

You suggestions please.

Thanks in advance.

EDIT: I've also tried Serial.println("Test"); but this also does not work.

Welcome to the world of protocols, OpenSource...

What is undoubtedly happening is that the reads on the computer are happening when the complete data isn't in the serial buffer on the PC.

You can counteract this a couple of ways (one which you touched upon)

  1. Pause and control data that way, ie send data, wait ten seconds, send data again.
  2. Mark the end of transmission (end of block, end of line...) so that the PC knows which block, line, etc goes with which.
  3. Mark the beginning of transmission
  4. Mark both beginning and endings.

I always like marking the beginning for cases when you are connecting a device to an already-communicating line. But, if we just watch for one or the other, it wouldn't be completely clear where we are when first connecting to that already running system.

You would have to ensure the serial lines were quiet, or we would be out of sync and miss part of the first packet.

(Imagine starting reading the message: don't delete that file after plugging in after the word 'don't')

Here is what I would do, for a simple protocol marking the beginning and ending of the block:

on the arduino:

Serial.print("!");
Serial.print("Something to send.\r\n");

on the Pc, here is the pseudo code: (I will leave it for you to code up...)

  • check our local data buffer. if there isn't a '\r\n' there, continue.
  • read the serial buffer, append any data there to our local buffer
  • check our local data buffer. if there is a '!', we have a block.
  • do whatever with the block of data between '!' and '\r\n'
  • remove the block from our local buffer, and go back to the top

Hi Spinlock,

On your advice I've marked the beginning and ending of my message:

Start :
End:

Example:
Serial.print("It's Alive....Alive")

This works great for sending data from the arduino to my PC.

Pseudo code (VB .NET Alert!!):

if ComPort.Readto("<START>") is true then 'Read Buffer until start

Msg = Comport.Readto("<END>") 'The characters are read until the footer (<END>) is reached.

end if

TextBox.Text = Msg

But I would also like to send data back from the PC to the arduino.

Now I send this message back to my Arduino: KeepRocking

I've seen this site that describes the use of a Header (NameBright - Coming Soon)

The problem is in the example there only one character is used as header and no footer is used. and the message length is variable.

Is it possible to read multiple characters?

How to deal with multiple characters is you header and footer? Any ideas?

Thanks in advance :slight_smile:

hello ...

hum hum .. why don't you use just one single character to define your start and ending of emssage .. there are pelnty of char that are never used (|,#,*). You'll save some bandwith at the same time ....

Sure... It isn't very optimal for a micro device to do it this way, but a quick and dirty way would be to buffer the data and scan it for the tags. (strnstr is used here, but you could write your own if you didn't want the linker to pull it in.)

// adjust this to the max.  Remember, limited Ram on the device!
#define MAX_MESSAGE_SIZE 512
char buffer[512];
int iBuffOffset=0;

void loop()
{
     while(Serial.available()>0)
     {
         // append to our buffer.
         buffer[iBuffOffset++] = Serial.read();
         //check for overflow
         if(iBuffOffset==MAX_MESSAGE_SIZE)
         {  
              Serial.print("<START>OVERFLOW<END>");
              iBuffOffset =0;
              return;
         }

         //check the buffer for the end tag.  We don't need to 
         // do this every character, just when we have more than
         // our start+end tags and something in between.
         // Adjust the 12 if you change START or END to something else.
         if(iBuffOffset>12)
         {
              char *pEnding = strnstr(buffer, "<END>", iBuffOffset);
              char *pBeginning = strnstr(buffer, "<START>", iBuffOffset);
              if(pEnding!=NULL && pBeginning!=NULL)
              {
                     // move over our start tag.
                     pBeginning+=7;
                     *pEnding = NULL;
                     doDataHandler(pBeginning);
                     iBuffOffset=0;
               }
               // handle case where End is available, but no start.  I would just reset the buffer.
               else if(pEnding!=NULL && pBeginning!=NULL)
                            iBuffOffset=0;
          }
     }
}

void doDataHandler(char *pData)
{
      // pData is what was between the start and end tags.
}

Wow thats fast!! :slight_smile:

Well I can use a single character as header and footer but I like the "<>"style.

I'll reconsider.

BTW: What do you mean with limited ram? That storing much characters consumes a lot of Ram? Well, the length will not exceed 20 characters :slight_smile:

Thanks.

Yes, the sample I wrote is a bit of a waste in terms of usage of memory and performance - and won't handle data with either or in it...

To make it faster, you could create your own strlen that would search for start and end in one pass through the buffer instead of two calls using strnstr.

You could just as easily take the code and modify it for something else like '!S', '!E' too to make things smaller and more efficient.

Just another question:

What is the optimal way to do this? Use a single character header and footer?

Is there a programmers reference (maybe a PDF file) for the Arduino Language? With example ect... I am having some difficulties with declaring (For Next, Do Loop) ect...

I know a little VB and the basic structure is the same but I keep making a lot of stupid errors...

EDIT: I get a error with: strnstr?

What is the optimal way to do this? Use a single character header and footer?

That depends on your data, speed requirements, and a ton of other things, but obviously the less you can put in the 'envelope' around your data, the best.

Some protocols first send 'size of transmission' and then the data, without header or footer. Some (like HTTP) use a blank line to indicate header vs data.

Is there a programmers reference (maybe a PDF file) for the Arduino Language? With example ect... I am having some difficulties with declaring ect...

Well, C and then C++ is a good place to start, but there are tons of samples on the http://arduino.cc/en/Tutorial/Foundations site. Ladyada has a good tutorial Arduino Tutorial - Learn electronics and microcontrollers using Arduino!

... and of course- there are tons of people doing cool things in the exhibition area of this forum, you can check out their code too...

Ok Thanks,

Last question (for now)

What is: strnstr in your example? I get a error when verifying?

What is *pEnding? You declare it as a char but in the if statement you use pEnding without the *

And how do I print back the received data between the header and footer back to the PC for example

serial.print(pData)?

I think that you can include it:

#include<avr\string.h>

Maybe that was a bad 'too-complex' example for a newer C programmer , but here is the short form on pointers.

char *pEnding in my example is a pointer to a char. The variable doesn't contain a value, just a way to get a value somewhere in memory that behaves like a char.

When I set it to the result of strnstr, which returns a NULL pointer (a pointer pointing at memory address 0) when it cannot find the string, or a pointer to where the second argument to strnstr when it is found.

So, when I do a pBeginning++, I move the actual pointer forward. If it was pointing to the E in "HELLO", it would point to the L after this call.

However, when I use it like this:
*pEnding =NULL;
The value that pEnding is pointing to is set to NULL. If *pEnding was pointing to the E in Hello, and if you printed it using Serial.print, Hello would be truncated to just H (because print stops printing when it sees character 0, or NULL (same thing)).

Now I get this error:

24: error: avr\string.h: No such file or directory In function 'void loop()':
Bad error line: -4

Is it some kind of library?

my mistake - I see now that strnstr isn't in the avr/arduino "bag of tricks."

I should have checked that first.

you can add this code to your sketch then:

char * strnstr (
        const char * str1,
        const char * str2, 
        int Length
        )
{
        char *cp = (char *) str1;
        char *s1, *s2;

        if ( !*str2 )
            return((char *)str1);

        while (*cp && Length-->0)
        {
                s1 = cp;
                s2 = (char *) str2;

                while ( *s1 && *s2 && !(*s1-*s2) && Length>0)
                {
                        s1++, s2++;
                        Length--;
                 }
                
                if(Length==0)
return NULL;

                if (!*s2)
                        return(cp);

                cp++;
        }

        return(NULL);

}

Wow..... Thats to difficult for me.

Just a quick summary:

It is much more easy to only use a header (single char). The advantage is that you don't have to fill an array (saves RAM), but just read the characters one by one using the serial.read() until the header passes (using the example from this site).

Correct me if I am wrong :slight_smile:

Thanks a lot.

Okay, I made this code (using a example):

int ledPin = 13;     // LED connected to digital pin 13

void setup(){
  pinMode(ledPin, OUTPUT);      // sets the digital pin as output
  Serial.begin(9600);
}

void loop()
{  
  if (nextByte() == 126) {        // header byte ('~' character)
    char charIn = 0;
    byte i = 0;
    while (charIn != 126) {  // wait for header byte again 
      charIn = nextByte();
      stringIn[i] = charIn;
      i += 1;
     }                       // stringIn is finished
     for (int j=i-1; j<16; j++) {
       stringIn[j] = ' ';    // null out string
     }
     Serial.print(stringIn);
     delay(10);
    }
}

byte nextByte() {
  while(1) {
    if(Serial.available() > 0) {
      byte b =  Serial.read();
      return b;
    }
  }
}

The problem is: stringIn[j]

in the example this is declared as:

char stringIn[] = {' ', ' ', ' ', ' ', ' ', 
                   ' ', ' ', ' ', ' ', ' ',
                   ' ', ' ', ' ', ' ', ' ', ' '};

But I use variable message length so I would like to solve this a different way. Any suggestions?

I am getting there :slight_smile:

Got this great PDF : http://cslibrary.stanford.edu/101/EssentialC.pdf

Got it finally working (with some small giltches), many thanks to AlphaBeta:

//constants
const byte INPUT_MAX_LENGTH = 25 + 1 + 1 + 1;//20 plus header plus footer plus terminating zero

const char HEADER = '~';
const char FOOTER = '#';

char stringIn[INPUT_MAX_LENGTH] = {'\0'};

int ledPin = 13;     // LED connected to digital pin 13


void setup(){
      pinMode(ledPin, OUTPUT);      // sets the digital pin as output
      Serial.begin(9600);
}
 void loop()
{  
      if (nextByte() == HEADER) {      
      
            byte i = 0;//index of char array
            
            stringIn[i++] = HEADER; //add the header to the char array
            
            char charIn = 0;
            
            do { 
                  charIn = nextByte();
                  stringIn[i++] = charIn;
            } while ( charIn != FOOTER && i<INPUT_MAX_LENGTH-1); //we can not exceed the MAX_LENGTH
            
 for (int j=i; j<INPUT_MAX_LENGTH; j++) {
                  stringIn[j] = '\0';    // null out string
            }
            Serial.print(stringIn);

            //delay(10);

                Serial.flush();
      }
}
 byte nextByte() {
      while(1) {
            if(Serial.available() > 0) {
                  byte b =  Serial.read();
                  return b;
            }
      }
}

Ok just a few things:

  1. How can I empty a array? stringIn[j] is still filled with data at he end of the cycle (is it necessary to empty it?)

  2. When I send a lot of commands with small intervals, some of the data gets lost (does not reach the PC). Is there software available that makes the transport of data going trough a serial port visible?

How to solve this? Increase the BaudRate?, I already inserted a Serial.Flush() to flush the buffer every cycle.

Thanks in advance :slight_smile:

Got it finally working (with some small giltches), many thanks to AlphaBeta:

//constants
const byte INPUT_MAX_LENGTH = 25 + 1 + 1 + 1;//20 plus header plus footer plus terminating zero

const char HEADER = '~';
const char FOOTER = '#';

char stringIn[INPUT_MAX_LENGTH] = {'\0'};

int ledPin = 13;     // LED connected to digital pin 13


void setup(){
      pinMode(ledPin, OUTPUT);      // sets the digital pin as output
      Serial.begin(9600);
}
 void loop()
{  
      if (nextByte() == HEADER) {      
      
            byte i = 0;//index of char array
            
            stringIn[i++] = HEADER; //add the header to the char array
            
            char charIn = 0;
            
            do { 
                  charIn = nextByte();
                  stringIn[i++] = charIn;
            } while ( charIn != FOOTER && i<INPUT_MAX_LENGTH-1); //we can not exceed the MAX_LENGTH
            
 for (int j=i; j<INPUT_MAX_LENGTH; j++) {
                  stringIn[j] = '\0';    // null out string
            }
            Serial.print(stringIn);

            //delay(10);

                Serial.flush();
      }
}
 byte nextByte() {
      while(1) {
            if(Serial.available() > 0) {
                  byte b =  Serial.read();
                  return b;
            }
      }
}

Ok just a few things:

  1. How can I empty a array? stringIn[j] is still filled with data at he end of the cycle (is it necessary to empty it?)

  2. When I send a lot of commands with small intervals, some of the data gets lost (does not reach the PC).

Is there software available that makes the transport of data going trough a serial port visible? I already have the Arduino communication trough the serial port with a small VB program. So I would like to place the (snifffer?)software in series with the excising connection

How to solve this? Increase the BaudRate?, I already inserted a Serial.Flush() to flush the buffer every cycle.

Thanks in advance :slight_smile:

for (int j=i; j<INPUT_MAX_LENGTH; j++) {
                  stringIn[j] = '\0';    // null out string
            }

A simple

stringIn[i] = '\0';

Would do the trick without the loop, and answers question 1).

Thanks AWOL :slight_smile:

I would like to compare the content of the stringIn with :

const char BtnU[] = "~ButtonUp#";

I am doing this the following way:

if (stringIn == BtnU) {
      Serial.print(stringIn);
    }

But this does not work... (and stringIn in = ~ButtonUp#)

What am I doing wrong?

And How do I add up strings?

For example Serial.print("Hello"+ "World") generates a error?

Thanks in advance

OpenSource,

There are library routines to do the things you want to do. See avr-libc: <string.h>: Strings.

For example, to compare two strings:

if (strcmp(BtnU,stringIn) == 0)  /* strings match */
  Serial.print(stringIn);

To append one string to another:

void
sayhello(char *username)
{
    char newstr[100];

    strcpy(newstr,"Hello, ");
    strcat(newstr,username);
    Serial.print(newstr);
}

Of course, you must always be sure that the destination string is big enough to hold the string you are building. For example, in the example above, if 'username' had 100 characters, data would be written past the end of 'newstr' and other variables in your program might be changed. That's a bad thing. You can use 'strncpy()' to prevent this.

-Mike