Receiving multi-character serial data

Hi there

Im struggling to receive serial messages made up from more that one ascii character on the arduino.

Works fine, from the arduino to the PC running a python, but i cant the the other direction working. Is it even possible?

the goal is to set on of several analog pins to a custom value

pin 11 to 50% power
or pin 9 to 20% power

Kind regards
Theron Burger

Of course it is possible - if you post your code (use the # code key above), we can probably help.

Thanks for the quick reply

The arduino code, in general, is my problem.

Here is a stripped down version of the python code

import serial

ser = serial.Serial('COM12')


while True:
    pin = raw_input('Please choose and output pin: ')
    val = raw_input('Please enter a value : ')
    
    message = pin+':'+val
    
    ser.write(message)
    
    print message

for the arduino side, here is some psuedo code

  //if data exists
  if (Serial.available() > 0) {
    //grab it
    data = Serial.read();

    //seperate the data back into pin and value
    pinString = data[0:2]
    valString = data[3:5]
    
    //Convert the strings to numbers
    pinNum = str2int(pinString)
    valNum = str2int(valString)
    
    digitalWrite(pinNum,valNum)
  }

Am I being silly to insist on sending it all through in one go?
splitting it seems a little simpler now in hindsight
(My arduino coding skills leve much to be desired)

//Multiple Output Analog Control

int data; 
int pin = 0;
int val = 0;

void setup() {
  // initialize serial communication:
  Serial.begin(9600);

}

void loop() {
  //if data exists
  if (Serial.available() > 0) {
    //grab it
    data = Serial.read();
      pin = data;
      
      //Wait for the next packet of data
      delay(50);
        //What sort of delays would one use here?
      
      if (Serial.available() > 0) {
        data = Serial.read();
        val = data;
        }
    
    if ((pin <= 11) and (pin >= 9) and (val != 0)){
      analogWrite(pin, val);
      }
       
  }
}

How about this:

import serial

ser = serial.Serial('COM12')


while True:
    pin = raw_input('Please choose and output pin: ')
    val = raw_input('Please enter a value : ')
    
    message = '!' + pin+':' + val + '

I am assuming of course that the data is written out in decimal by Python (forgive me for not being all that familiar with it...) Hopefully someone else can advise as to this validity of this code...

FYI : The signals are being marked, so we know the beginning(!), middle(:slight_smile: and end of transmission($) in the above code. The arduino won't process until it sees a '$'.

And the arduino sketch to handle it:

int iState =0;
int iPin=0;
int iValue=0;

void loop()
{
   while(Serial.available()>0)
   {
      c = Serial.read();
      if(c=='!')
     {
         iState=0;
         iPin=0;
         iValue=0;
      }
      else if(c=='

Does this make sense?
   
   ser.write(message)
   
   print message


I am assuming of course that the data is written out in decimal by Python (forgive me for not being all that familiar with it...) Hopefully someone else can advise as to this validity of this code...

FYI : The signals are being marked, so we know the beginning(!), middle(:) and end of transmission($) in the above code. The arduino won't process until it sees a '$'.

And the arduino sketch to handle it:

§DISCOURSE_HOISTED_CODE_1§


Does this make sense?)
      {
         if(iPin<=11 && iPin>=9)
           analogWrite(iPin, iValue);
         iState=0;
         iPin=0;
         iValue=0;
      }
      else if (c==':')
         iState=1;
      else if(c>='0' && c<='9')
      {
         if(iState==0) // pin
         {
             iPin*=10;
             iPin+=c;
         }
         else  // value
         {
             iValue*=10;
             iValue+=c;
         }
      }
      else
     {
        // invalid digit.
     }
   }
}

Does this make sense?
   
   ser.write(message)
   
   print message


I am assuming of course that the data is written out in decimal by Python (forgive me for not being all that familiar with it...) Hopefully someone else can advise as to this validity of this code...

FYI : The signals are being marked, so we know the beginning(!), middle(:) and end of transmission($) in the above code. The arduino won't process until it sees a '$'.

And the arduino sketch to handle it:

§DISCOURSE_HOISTED_CODE_1§


Does this make sense?

Ah, right, now its all beginning to make sense!

correct me if I'm wrong

The arduino doesn't receive whole chunks of data, it receive it character by character.

so... say there is a buffer, that python has dumped !13:220$

then as the while(Serial.available()>0) loop runs, it dumps character by character into c

so if you wanted the whole word, you would go

word = word + c

Ok, so how does this look too you
So, assuming you only send the pin and the value, ie 13220

//Multiple Output Analog Control


int  pin       = 0;
int  val       = 0;
int  i         = 0;
char message[] = "00000";

void setup() {
  // initialize serial communication:
  Serial.begin(9600);
}

void loop() {
  i = 0;
  //if data exists
  while (Serial.available() > 0) {
    //store it in the message array char by char
    message[i] = Serial.read();
    //increase the counter by one
    i = i + 1;
    
    if (i = 4){
      //Get the values and convet them to integers
      pin = int(message[0])*10  + int(message[1]);
      val = int(message[2])*100 + int(message[3])*10 + int(message[4]);
      //write them to the pin
      analogWrite(pin, val);
    }
  }
}

Granted there is no error checking etc. Pity i left the Arduino at work and can only test in tomorrow morning :frowning:
Thanks for all your help so far. I think this community has something going for it :slight_smile:

The arduino doesn't receive whole chunks of data, it receive it character by character.

Correct. It also doesn't know when you started or stopped sending. it simply reads the next out of the internal UART buffer (if using hardware Serial...) [That is key. You aren't waiting to read - it may already be in the buffer, if the Serial class is using something that has one or emulates one...)

so... say there is a buffer, that python has dumped !13:220$

then as the while(Serial.available()>0)  loop runs, it dumps character by character into c

so if you wanted the whole word, you would go

word = word + c

Well, Yes - but not literally. Your code sample (in your last post) is more correct than just word=word+c.

So, assuming you only send the pin and the value, ie 13220

I don't see a problem without a separator in the beginning, but you do have to ensure that every pin value you send is a two digit number and values, 3 digits with that code - and that you don't send too many commands at a time.

That is also the reason I added the start characters to mark beginning in my sample which prevents this data from coming in:

15220152015220

If the code missed a digit (because the buffer was full, or the arduino was plugged in half way through a communication, or a usb hub was busy, etc... your code could have written pin 5 with a value of 220, instead of pin 15 with 220.

If you are manually entering in 05220[enter] and not ever planning to automate it, your code should work more-or-less ok.

(except this line:)

if (i = 4){
      //Get the values and convet them to integers

Unfortunately it always will execute that line.
Try i==4 instead. And, don't forget to reset i to 0 afterwards of course, our the second time you will clobber stack.

:wink:

Ah, I see, the buffer may not only be holding the message you want, and you cant clear it on the arduino side, as you don't know what the sender side is doing.

Ok, so i think I'v got a clearer understanding of serial communication now.

Now its just my Arduino language that needs work

Thanks for all your help
Ill post the program when its done :slight_smile:

Sorry for the long delay, the arduino projects are low priority ones :slight_smile:

Ok, so i went a little against spinlock's recommendation here in terms of error checking, but I'm assuming all commands are formatted properly

/*
Serial Multiple Output Analog Control V001
By Theron Burger
Rossum Robotics
2009

With alot of help from spinlock

Commands sould be sent through in the format:
!pinvalue$
pin being 2 decimal digits, IE 09 or 11
value being 3 decimal digits, IE 009 or 107 or 255
*/

int  iState = 0;  //Place holder for marking the position of the incomming charactors
int  iPin   = 0;  //Place holder for the pin number
int  iValue = 0;  //Place holder for the value we want to right to the pin
char c      = '0';

void setup(){
  Serial.begin(9600);
  delay(1000);
  Serial.println("System Online!");
  Serial.println('0',DEC);
}
void loop()
{
  while(Serial.available()>0){ //While there is stuff in the buffer
    c = Serial.read();  //pull out a single charactor
    if(c == '!'){  //If the charactor is a !, we'r at the start, so set everything to 0
      Serial.println("Got start charactor, begining process");
      iState  = 0;
      iPin    = 0;
      iValue  = 0;
      c = Serial.read();  //Grab the next charactor, it should be the first charactor of the pin
      iPin    += (c-48)*10;  //(c-48) is needed as decimal 0 is ASCII 48
      c = Serial.read();  //Grab the next charactor, it should be the second charactor of the pin
      iPin    += (c-48);
      c = Serial.read();  //Grab the next charactor, it should be the first charactor of the value
      iValue  += (c-48)*100;
      c = Serial.read();  //Grab the next charactor, it should be the second charactor of the value
      iValue  += (c-48)*10;
      c = Serial.read();  //Grab the next charactor, it should be the third charactor of the value
      iValue  += (c-48);
      }
    else if(c=='

It seems to work ok :slight_smile:
Thanks for all your help){  //If the charactor is a $, we'r at the end of the command, so write the values to the pins
     Serial.println("Got end charactor, writing to pin");
     //if(iPin>=9 && iPin<=11){  //If the pin is valid, IE an analog pin
       analogWrite(iPin, iValue);
       Serial.print("Pin : ");
       Serial.print(iPin);
       Serial.print("   Value : ");
       Serial.println(iValue);
       //}
     }
  }
}


It seems to work ok :)
Thanks for all your help

It seems to work ok

Right up to the point where it bites you on the bottom.

  while(Serial.available()>0){ //While there is stuff in the buffer
    c = Serial.read();  //pull out a single charactor
    if(c == '!'){  //If the charactor is a !, we'r at the start, so set everything to 0
      Serial.println("Got start charactor, begining process");
      iState  = 0;
      iPin    = 0;
      iValue  = 0;
      c = Serial.read();  //Grab the next charactor, it should be the first charactor of the pin

Up to the point where you hit "c = Serial.read ();" you're ok, but "Serial.available ()" may have returned "1", in which case your "c = Serial.read ();" will set "c" to -1.
The only thing that is stopping this happening is the "Serial.println ("Got start charactor, begining process");", which at 9600 baud is roughly equivalent to "delay (40);", giving you enough time to buffer the rest of the characters from the PC.

If you ever removed the "Serial.println ()", this would almost certainly happen.

You should always check to see if sufficient data is available before reading it.
You'd probably be better off writing a short function that takes as a parameter the number of characters you want to read, and returns an integer value of the number read from the serial port, taking care of making sure you have sufficient data. Shouldn't be more than about ten or twelve lines.

Awol, We're having exactly the issue you describe in our stepper control code.

I understand why it will fail without the delay but I'm trying to understand why it needs a >20ms delay to work correctly. if its 9600bps thats (1/9600)*8 = 0.00083s <1Ms, for the time it takes a byte to arrive at the arduino? increasing the baud rate for serial comms reduces the length of the delay needed but that obviously isn't a fix, just confirms that our code sucks slighly faster at higher baud rates.

So the code in void loop() is running faster than that hence getting a -1 as it hits between bytes but why does it need more than 20ms delay to get 5 chars. In our case 5 chars is the smallest string we send, longest string is probably a buffer full, we don't check, mainly as its not that important as our release software will be usually sending just a single command+optional data string, but random length multiple command strings are very useful while I debug and could springboard another project that they would be very useful in.

Would it be legitimate for me to use something like:

if (Serial.available() > 4)

before we start to process the serial buffer? or am I hoping that by the time its processed the command string, the data portion of the command (if it exists) will have arrived?

Our command strings take the following form, where nnnnn = 0 to 16384

:POS:nnnnn; 
:SPD:nnnnn; 
:STP:nnnnn;
:LMT:nnnnn;
:IN1:n; // already padded to keep the commands to 3 bytes
:OUT:n;

IN1 and OUT don't actually need a value unless you are sending a multiple command string like this:

:spd:1000;:lmt:16000;:stp:500;:in1:0;:stp:50;:out:0;:stp:60;:out:0;

We know why in and out fail with the 0; ommited, because we still check for a value which kills the string as it reads the next command in while it hunts for a ; to finish the last command, that's fine we'll fix that in a bit by moving the call to the 'get the value for the command' function into the commands that actually require a value.

some of our code:

void loop() {
  
  
  int bCommandReady = false;

  //If There is information in the Serial buffer read it in and start the Build command subroutine
  if (usingSerial && Serial.available() > 0) {  // I expect to change this to 
    // read the incoming byte:
    incomingByte = Serial.read();
    /* Build a new command. */
    bCommandReady = cliBuildCommand(incomingByte);
  }


delay(100);  // with this lower than 20ms our code goes sporadic, the lower you go it fails (as explained by awol)
  //If there is a command in the buffer then run the process command subroutine
  if (bCommandReady == true) {
    bCommandReady = false; // reset the command ready flag
    cliProcessCommand(); // run the command
  }

  if (UPDATE){
    UPDATE=false;
    FocusPrintStepsFun(); //Print the number of steps
    FocusPrintPositionFun(); //Print the Position
    SerialDATAFun();  // debug mainly, gives detailed information about the current state of the machine

  }

}

and the command/data building functions:

//Process Command. This searches the command table to see if the command exits if it does then the required subroutine is run
void cliProcessCommand(void)
{
  int bCommandFound = false;
  int idx;

  /* Convert the parameter to an integer value. 
   * If the parameter is emplty, gParamValue becomes 0. */
  gParamValue = strtol(gParamBuffer, NULL, 0);

  /* Search for the command in the command table until it is found or
   * the end of the table is reached. If the command is found, break
   * out of the loop. */
  for (idx = 0; gCommandTable[idx].name != NULL; idx++) {
    if (strcmp(gCommandTable[idx].name, gCommandBuffer) == 0) {
      bCommandFound = true;
      break;
    }
  }

  /* If the command was found, call the command function. Otherwise,
   * output an error message. */
  if (bCommandFound == true) {
    (*gCommandTable[idx].function)();
  }
}


//When data is in the Serial buffer this subroutine is run and the information put into a command buffer.
// The character : is used to define the end of a Command string and the start of the parameter string
// The character ; is used to define the end of the Parameter string
int cliBuildCommand(char nextChar) {
  static uint8_t idx = 0; //index for command buffer
  static uint8_t idx2 = 0; //index for parameter buffer
  int loopchk = 0;

  nextChar = Serial.read();
  do
  {
    gCommandBuffer[idx] = TO_UPPER(nextChar);
    idx++;
    loopchk=loopchk+1;
    nextChar = Serial.read();

  } 
  while ((nextChar != ':') && (loopchk < 6));

  loopchk=0;

  nextChar = Serial.read();

  do
  {

    gParamBuffer[idx2] = nextChar;
    idx2++;
    loopchk=loopchk+1;
    nextChar = Serial.read();
  } 
  while ((nextChar != ';')&& (loopchk < 5));



  gCommandBuffer[idx] = '\0';
  gParamBuffer[idx2] = '\0';
  idx = 0;
  idx2 = 0;

  return true;
}