Problem with serial communication

hey guys

I need to send a string to Arduino from Python using serial communication. The string is like <-4567-> for example, where < and - are delimiters to be used when I'm parsing the data.

However, I checked what it is being sent to Arduino and I obtain only <. I think it's something with the syntax in Python or how i approached the incoming data to be read.

Below are the codes:

Python 1 : this part is where I write to a list testData1 and testData2 the string that I want to send.

serPort = "COM6"
baudRate = 57600
ser = serial.Serial(serPort, baudRate)
time.sleep(2)


serPort1 = "COM5"
baudRate = 57600
ser1 = serial.Serial(serPort1, baudRate)
time.sleep(2)

startMarker = 60
endMarker = 62

ser.close()
ser1.close()


tau1 = []
tau2 = []

for i in range(15620,15622,1):  
    
    testData1 = []
    tau1.append(i)
    sData1 = '<-'+str(i)+'->'
    testData1.append(str(sData1).encode('ascii'))
    runTest1(testData1)
    
    testData2 = []
    tau2.append(i)
    sData2 = '<-'+str(i)+'->'
    testData2.append(str(sData2).encode('ascii'))
    runTest2(testData2)

Python 2: this part is where the functions sendToArduino and runTest are written:

def sendToArduino1(teststr):
    ser.write(teststr)
 

def sendToArduino2(teststr):
    ser1.write(teststr)
  
def runTest1(td):
    
    teststr = td
    sendToArduino1(teststr)
    time.sleep(1)
    
#    numLoops = len(testData1)
#    n = 0
#    while n < numLoops:
#        teststr = testData1[n]
#        sendToArduino1(teststr)
#        n = n+1
            
#    time.sleep(1)
#    
def runTest2(td):
    
    teststr = td
    sendToArduino2(teststr)
    time.sleep(1)

Arduino: How I approach on Arduino

const byte buffSize = 10;
char inputBuffer[buffSize];
char tempChars[buffSize];
char messageFromPC[buffSize] = {0};
boolean newDatafromPc = false;


void loop() {

  receivedData();
  if(newDatafromPc == true){
    strcpy(tempChars,inputBuffer);
    noInterrupts();
    parseData();
    interrupts();
    newDatafromPc = false;
  }

}

void receivedData(){
  static boolean readInProgress = false;
  static byte bytesRecvd = 0;
  char startMarker = '<';
  char endMarker = '>';
  char x;
    while (Serial.available() > 0 && newDatafromPc == false) {
        
       x = Serial.read();
       Serial.write(x);  // just to verify on python what was sent
       
        if (readInProgress == true) {
            if (x != endMarker) {
                inputBuffer[bytesRecvd] = x;
                bytesRecvd++;
                if (bytesRecvd >= buffSize) {
                    bytesRecvd = buffSize - 1;
                }
            }
            else {
                inputBuffer[bytesRecvd] = '\0'; // terminate the string
                readInProgress = false;
                bytesRecvd = 0;
                newDatafromPc = true;
            }
        }

        else if (x == startMarker) {
            readInProgress = true;
        }
        
    }
}



void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(tempChars,"-");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, "-"); // this continues where the previous call left off
  delayPeriod = atoi(strtokIndx);     // convert this part to an integer

}

What I want is just the value inside the string. For example, <-4567-> I want 4567.
Could anyone help me?

Thanks

Have a look at this simple Python - Arduino demo

Also see Serial Input Basics - simple reliable ways to receive data.

...R

    noInterrupts();
    parseData();
    interrupts();

Stop turning interrupts off and on indiscriminately. You clearly have no idea why you are doing this, so stop doing it.

I do it because it requires a special treatment since it's a 16-bit data and as a volatile variable, I have to ensure this protection. Help me then instead of complaining. Learning is based on erros and I'm not using this forum to waste people time.

I've mentioned more than once that the critical section (the part where interrupts are disabled) is ONLY for copying data.

char copyOfReceivedData[bufferSize];
noInterrupts();
memcmp(copyOfReceivedData, receivedData, bufferSize);
interrupts();
parseData(copyOfReceivedData);

Now, you can rewrite parseData() to take an argument - the array containing the data to parse - instead of using a hardcoded array name.

I don't understand why noInterrupts() is being used at all since there is no Interrupt Service Routine that might change a value.

The code in the function receivedData() looks very like the code in recvWithStartEndMarkers() in Serial Input Basics. I wonder why the name was changed?

In any case there is no instance of noInterrupts() in my tutorial and it works perfectly well.

...R

@PaulS I checked my code and I protected the part responsible for copying the data like below:

void loop() {

  receivedData();
  if(newDatafromPc == true){
    noInterrupts();
    strcpy(tempChars,inputBuffer);
    interrupts();
    parseData();
    newDatafromPc = false;
  }

}

I used the examples of Robin2 to establish the communication. I had to switch atoi() to atol() because the variable delayPeriod is a long int.

The problem is remaining: i'm not able to delay my pulse in the amount of time I want and as @gfvalvo said in a thread before, probably the FIFO structure is not a good solution for me. I'm trying to figure out how to do it. Do you have an idea how i could solve this?

Resuming the problem: I have 2 Arduinos connected to 2 separate oscillators. The oscillations which depends on the discharge of a capacitor occur at a rate of 1Hz. What I need to do: after my Arduino capture this discharge(spike on the oscilloscope), I need to generate a pulse of 60us width and this pulse must be delayed by certain amount of time (this value comes from Python). Until 1s delay I am on the first period, but if I choose around 1,5s, we are on the second period and my system must ensure that it can store previous spikes in orde to generate the delayed pulse later.

Crespo94:
I used the examples of Robin2 to establish the communication.

So why have you introduced noInterrupts() ?

...R

Due to a volatile variable and because of its size, it might be changed during the process. Why would it not be correct to do that ?

@Robin2, if I have data coming from Python like <-VALUE->, with 2 delimiters < and -, by using that function recvWithStartEndMarkers(), am I able to extract the value with parseData()?

I also implemented the python code that you provided me here.

You are misunderstanding how strtok() works. The first time you call strtok(), it searches for the first character that is NOT "-" and sets that as the beginning of the string, then searches for the next character that is "-", and sets that as the end of the string, so a single call to strtok() returns a string containing the number deliminated by the hyphens. A simple Serial.println(strtokIndx) to see what the string looked like could have shown you this. Your function should look like this:

void parseData() {
  // split the data into its parts
  char * strtokIndx; // this is used by strtok() as an index
  strtokIndx = strtok(tempChars, "-");     // get the first part - the string
  delayPeriod = atoi(strtokIndx);     // convert this part to an integer
}

As others have mentioned, get rid of the noInterrupt(), you are not (at least in the code you have shown) using the variable inside an ISR, so not reason to protect it from interrupts.

Thank you @david_2018! Should I use atoi or atol as delayPeriod is a uint_16 variable ?

I still dont get the result I want. Maybe it's something with timers. From what I could study, my setup is correct. Could you help me if you know something about it ?

Thanks

startMarker = '<'
endMarker = '>'
dlm1 = '-'
dlm2 = '-'

#SERIAL --------------------------------------------------------------
def sendToArduinos(stringToSend):
    
    global startMarker, dlm1, dlm2, endMarker, ser1, ser2
    
    stringWithMarkers = (startMarker)
    stringWithMarkers += (dlm1)
    stringWithMarkers += stringToSend
    stringWithMarkers += (dlm2)
    stringWithMarkers += (endMarker)
    
    ser1.write(stringWithMarkers.encode('utf-8'))
    ser2.write(stringWithMarkers.encode('utf-8'))
    
    time.sleep(1)

this is how I send data to Arduino. The value inside the function sendToArduinos is like "14200" for example. But the whole data is sent like <-14200->.

Referring to Replies #8 and #9 ...

I have no idea why you have a volatile variable but I cannot think of any reason why it should have any bearing on my serial code.

Yes you can parse the data that is received using the function recvWithStartEndMarkers() - have you not looked at the parse example in Serial Input Basics ? Of course you may need to amend the details to match the sort of data you are sending.

You need to post BOTH your Python program and your Arduino program (complete programs, not snippets) - hopefully both of them are very short. Also post a sample of the output from your Arduino program.

...R

Below are the codes:

Python:

from __future__ import division
from __future__ import absolute_import
from __future__ import print_function
from __future__ import unicode_literals
from   scipy    import interpolate
import time
import serial
from picoscope import ps3000a
import numpy as np
import glob
import scipy.io as scio
import scipy.cluster as sclus
import scipy.signal as signal
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
from threading import Thread
import threading

#VARIABLES

startMarker = '<'
endMarker = '>'
dlm1 = '-'
dlm2 = '-'

#SERIAL --------------------------------------------------------------
def sendToArduinos(stringToSend):
    
    global startMarker, dlm1, dlm2, endMarker, ser1, ser2
    
    stringWithMarkers = (startMarker)
    stringWithMarkers += (dlm1)
    stringWithMarkers += stringToSend
    stringWithMarkers += (dlm2)
    stringWithMarkers += (endMarker)
    
    ser1.write(stringWithMarkers.encode('utf-8'))
    ser2.write(stringWithMarkers.encode('utf-8'))
    
    time.sleep(1)
    
    
def setupSerial1(baudRate, serialPortName):
    
    global ser1
    
    ser1 = serial.Serial(port = serialPortName, baudrate = baudRate)
    
    time.sleep(2)
    
    
def setupSerial2(baudRate, serialPortName):
    
    global ser2
    
    ser2 = serial.Serial(port = serialPortName, baudrate = baudRate)
    
    time.sleep(2)

#PROGRAM-----------------------------------
setupSerial1(57600, "COM6")
setupSerial2(57600, "COM5")

ser1.close()
ser2.close()

sendToArduinos("15625")

Arduino:

#include <Arduino.h>
#include <string.h>
#include <stdio.h>
#include <avr/io.h> 
#include <avr/interrupt.h>


const int16_t pulseWidth = 1;      //
const uint8_t logFifoSize = 3;
const uint8_t fifoSize = 1 << logFifoSize;
const uint8_t fifoMask = fifoSize - 1;
const uint8_t ICP1 = 8;      // Input capture pin

volatile uint16_t fifo[fifoSize];
volatile uint16_t delayPeriod; 
volatile uint8_t writeHead = 0;
volatile uint8_t readHead = 0;
volatile bool waitingForRising = false;
volatile bool waitingForFalling = false;

const byte buffSize = 10;
char inputBuffer[buffSize];
char tempChars[buffSize];
char messageFromPC[buffSize] = {0};
boolean newDatafromPc = false;


void setup() {
  pinMode(ICP1, INPUT);
  Serial.begin(57600);
  
  TCCR1B = (1 << ICES1);    // Stop counter, waveform generator Mode 0, capture input on rising edge of ICP - test
  TCCR1A = 0;         // waveform generator Mode 0
  TCNT1 = 0;          // reset counter
  OCR1A = 0xFFFF;       // park output compare value here for now
  TCCR1A = (1 << COM1A1);   // OC1A low on A-match, waveform generator Mode 0
  TCCR1C = (1 << FOC1A);    // Force A-match, OC1A Low -test - test
  TCCR1A = 0;         // disconnect OC1A
  PORTB &= ~(1 << PB1);   // PB1 (Pin 9) to 0 when switched to output
  DDRB |= (1 << DDB1);    // PB1 (Pin 9) as output
  TIMSK1 = (1 << ICIE1);    // interrupt on input captured - test
  TCCR1B |= (1 << CS10) | (1 << CS12) ;  // Start counter prescaler = 1024

}

void loop() {

  receivedData();
  if(newDatafromPc == true){
     strcpy(tempChars,inputBuffer);
     parseData();
     newDatafromPc = false;
  }

}

void receivedData(){
  static boolean readInProgress = false;
  static byte bytesRecvd = 0;
  char startMarker = '<';
  char endMarker = '>';
  char x;
    while (Serial.available() > 0 && newDatafromPc == false) {
        
       x = Serial.read();
             
       if (readInProgress == true) {
            if (x != endMarker) {
                inputBuffer[bytesRecvd] = x;
                bytesRecvd++;
                if (bytesRecvd >= buffSize) {
                    bytesRecvd = buffSize - 1;
                }
            }
            else {
                inputBuffer[bytesRecvd] = '\0'; // terminate the string
                readInProgress = false;
                bytesRecvd = 0;
                newDatafromPc = true;
            }
        }

        else if (x == startMarker) {
            readInProgress = true;
        }
        
    }
    
}

void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(tempChars,"-");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, "-"); // this continues where the previous call left off
  delayPeriod = atoi(strtokIndx);     // convert this part to an integer

}

ISR(TIMER1_CAPT_vect) {
  bool fifoEmpty = false;
  uint16_t timeStamp = ICR1; //moment of spike identification
  uint16_t risingEdgeTime = timeStamp + delayPeriod; 
 
  if (!waitingForRising && !waitingForFalling) {  // (readHead == writeHead) && counter value for rising edge
    fifoEmpty = true;
  }
  fifo[writeHead++] = risingEdgeTime;
  writeHead &= fifoMask;
  if (fifoEmpty) {
    readHead = writeHead;
    OCR1A = risingEdgeTime;           // set next compare time
    TIMSK1 |= (1 << OCIE1A);        // Enable COMPA on match
    TCCR1A = (1 << COM1A1) | (1 << COM1A0); // OC1A high on A-match - test
    waitingForRising = true;
    waitingForFalling = false;
  
  }
}

ISR(TIMER1_COMPA_vect) {
  if (waitingForRising) {
    OCR1A += pulseWidth;    // set time for output to go low
    TCCR1A = (1 << COM1A1);   // OC1A low on A-match - test
    waitingForRising = false;
    waitingForFalling = true;
    return;
  }

  if (waitingForFalling) {
    if (readHead == writeHead) {
      // FIFO is empty
      TIMSK1 &= ~(1 << OCIE1A); // disable interrupt on match
      TCCR1A = 0;         // disconnect OC1A
      waitingForFalling = false;
      waitingForRising = false;
      return;
    }
   
    // still events left in FIFO
    waitingForRising = true;
    waitingForFalling = false;
    OCR1A = fifo[readHead++];
    readHead &= fifoMask;
    TCCR1A = (1 << COM1A1) | (1 << COM1A0); // OC1A high on A-match - test
   
  }
}

On attachment is the figure. The red pulse should be delayed in 1s (15625 corresponds to 1s)from the yellow one (check delta value ) but it is 460ms.

Why have you closed the Serial ports

setupSerial1(57600, "COM6")
setupSerial2(57600, "COM5")

ser1.close()
ser2.close()

Why have you two serial connections?

...R

I have 2 Arduinos and I send the same data to both at the same time.

I don't compile all this code at once, but in parts. I only close the serial ports if I want to upload another code to the Arduinos

Crespo94:
I don't compile all this code at once, but in parts. I only close the serial ports if I want to upload another code to the Arduinos

That makes no sense in the context of the Python code you posted. Your code opens the ports and then immediately closes them.

Also, get the thing working with a single Arduino before trying anything more complicated.

...R

If you are actually sending data in the format of <-12345-> then get rid of the second strtok() in the parseData function.
The way it is written now, if you send <-12345-> messsageFromPC will contain 12345, and delayPeriod will be 0, because the 2nd strtok returns a pointer to an empty string.

void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(tempChars,"-");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, "-"); // this continues where the previous call left off
  delayPeriod = atoi(strtokIndx);     // convert this part to an integer

}