Speeding up my Slow Serial!?

Your receiver is expecting a '>'. You're not sending one any more.

As to looking at the output, can you temporarily send it to Serial instead so you can use the IDE serial monitor?

Ok, I’ve read Robin’s guide and decided to try his example 5 and have adapted it as such:

Sending code is now simply (executed every 10ms):

  Serial1.print("<");
  Serial1.print(SPEED_MPH);
  Serial1.print(", ");
  Serial1.print(RPM);
  Serial1.print(", ");
  Serial1.print(COOLANT_TEMP);
  Serial1.print(", ");
  Serial1.print(FUEL_LEVEL);
  Serial1.print(">");

The receiving code is now:
//Setup:

const byte numChars = 32;
char receivedChars[numChars];
char tempChars[numChars];        // temporary array for use when parsing

boolean newData = false;

//loop:

    recvWithStartEndMarkers();
    if (newData == true) {
        strcpy(tempChars, receivedChars);
            // this temporary copy is necessary to protect the original data
            //   because strtok() used in parseData() replaces the commas with \0
        parseData();
        showParsedData();
        newData = false;
    }

//voids:

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;

    while (Serial1.available() > 0 && newData == false) {
        rc = Serial1.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

//============

void parseData() {      // split the data into its parts

    char * strtokIndx; // this is used by strtok() as an index

    strtokIndx = strtok(tempChars,",");      // get the first part - the string
    SPEED_MPH = atoi(strtokIndx); // copy it to messageFromPC
 
    strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
    RPM = atoi(strtokIndx);     // convert this part to an integer
    
    strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
    COOLANT_TEMP = atoi(strtokIndx);     // convert this part to an integer
    
    strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
    FUEL_LEVEL = atoi(strtokIndx);     // convert this part to an integer
}

This actually works very well - much better than before with nice fluid operation. The only issue I now have is that if I switch to the RPM page on my display, I can see it is flickering ever so slightly. I think every once in a while an error value comes in perhaps. It was doing this more when the send code was set to 5ms intervals, and setting to 10ms mostly cured it, but if I go any higher, even to 11ms then nothing is read at all. I do not understand why?? Maybe the baud is too high… I’ll try that next

Update: slowing the baud makes it worse, but when I look at the serial monitor on the receiver, it looks like the code is tripping its self up and sometimes getting the values on the wrong slots...

RPM = 5220

Temp = 90

Fuel = 29.00

Speed = 172

RPM = 5220

Temp = 90

Fuel = 29.00

Speed = 172

RPM = 20

Temp = 43

Fuel = 29.00

Speed = 172

RPM = 52

Temp = 5220

Fuel = 90.00

Speed = 172

RPM = 5220

Temp = 90

Fuel = 29.00

Speed = 172

RPM = 5220

Temp = 90

Fuel = 2.00

Speed = 173

RPM = 5220

Temp = 90

Fuel = 29.00

You see here that on one occasion the RPM isn't fully read, and the fuel level too... What can I do to fix that?

May I suggest that you post the complete sender and complete receiver code?

Ok, I’ve read Robin’s guide and decided to try his example 5 and have adapted it as such:
This actually works very well - much better than before with nice fluid operation

Finally!. Getting rid of all the String manipulation and working with the character arrays is a big step in the right direction.

Sending code is now simply (executed every 10ms):

I’m not sure about how you are sending the Fuel value but it looks like you are sending

start ‘<’
stop ‘>’
3 commas ‘,’
14 numerical character

19 x 10 = 190 bits

at 9600 baud (bits per second) that takes approximately 20 ms.

You want to keep the baud rate as high as possible. Otherwise you will not be reading every message sent.

You see here that on one occasion the RPM isn’t fully read, and the fuel level too… What can I do to fix that?

If you are loosing data do to a noisy system, the way to deal with that is to use a checksum or CRC error check on the data, and only accept a valid transmission.

The only issue I now have is that if I switch to the RPM page on my display, I can see it is flickering ever so slightly

Can you please explain more about the display and the “flickering” issue. What type of display? Are you only printing changed elements? Does the flickering relate to the truncated “bad” data or is it more global?

It looked like some of the errors were related to the bigger integers (>100), so I decided to try splitting all of them down to 2 digit packets with limited success…

My send code is now:

void Send_Serial_Data() {

  int RPM1 = RPM / 100;
  int RPM2 = RPM - (RPM1 * 100);

  int SPEED1 = SPEED_MPH / 10;
  int SPEED2 = SPEED_MPH - (SPEED1 * 10);

  int COOLANT_TEMP1 = COOLANT_TEMP / 100;
  int COOLANT_TEMP2 = COOLANT_TEMP - (COOLANT_TEMP1 * 100);

  int FUEL_LEVEL1 = FUEL_LEVEL / 100;
  int FUEL_LEVEL2 = FUEL_LEVEL - (FUEL_LEVEL1 * 100);

  Serial1.print("<");
  Serial1.print(SPEED1);
  Serial1.print(", ");
  Serial1.print(SPEED2);
  Serial1.print(", ");
  Serial1.print(RPM1);
  Serial1.print(", ");
  Serial1.print(RPM2);
  Serial1.print(", ");
  Serial1.print(COOLANT_TEMP1);
  Serial1.print(", ");
  Serial1.print(COOLANT_TEMP2);
  Serial1.print(", ");
  Serial1.print(FUEL_LEVEL1);
  Serial1.print(", ");
  Serial1.print(FUEL_LEVEL2);
  Serial1.print(">");
}

Then the receive code is now the same as the last post, but I changed the parseData function to:

void parseData() {      // split the data into its parts

  char * strtokIndx; // this is used by strtok() as an index

  strtokIndx = strtok(tempChars, ",");     // get the first part - the string
  SPEED1 = atoi(strtokIndx); // copy it to messageFromPC

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  SPEED2 = atoi(strtokIndx);     // convert this part to an integer

  SPEED_MPH = (SPEED1 * 10) + SPEED2;

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  RPM1 = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  RPM2 = atoi(strtokIndx);     // convert this part to an integer

  RPM = (RPM1 * 100) + RPM2;

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  COOLANT_TEMP1 = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  COOLANT_TEMP2 = atoi(strtokIndx);     // convert this part to an integer

  COOLANT_TEMP = ((COOLANT_TEMP1 * 100) + COOLANT_TEMP2) / 10;

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  FUEL_LEVEL1 = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  FUEL_LEVEL2 = atoi(strtokIndx);     // convert this part to an integer

  FUEL_LEVEL = ((FUEL_LEVEL1 * 100) + FUEL_LEVEL2) / 10;


}

This seems to work pretty well but still the display is flickering - it’s a 128x32 pixel OLED display running on i2C. I don’t think it’s the display though, it’s the data. If I use the following function to display the incoming messages in the Serial monitor:

void showParsedData() {
  Serial.print("<");
  Serial.print(SPEED_MPH);
  Serial.print(", ");
  Serial.print(RPM);
  Serial.print(", ");
  Serial.print(COOLANT_TEMP);
  Serial.print(", ");
  Serial.print(FUEL_LEVEL);
  Serial.println(">");

}

Then I get this stream of data - notice a couple of missed values where the message has truncated slightly. This is running at 250000 baud.

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 815, 601, 990.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<151, 9902, 790, 0.00>

<69, 1560, 10, 156.00>

<69, 1560, 110, 91.00>

<69, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<141, 9902, 790, 0.00>

<68, 1560, 10, 156.00>

<68, 1560, 110, 81.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 915, 601, 990.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 7900, 0, 0.00>

<68, 1560, 119, 20.00>

<68, 1500, 81, 601.00>

<68, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<159, 279, 0, 0.00>

<69, 1560, 110, 91.00>

<69, 1560, 0, 156.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<145, 6011, 990, 790.00>

<68, 108, 156, 119.00>

<69, 1560, 119, 20.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<68, 815, 601, 990.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 1560, 119, 27.00>

<68, 915, 601, 990.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<69, 1560, 119, 27.00>

<68, 815, 601, 990.00>

Go upstream a bit: try printing the string you accumulated on the receiver before you parsed it.

I think you're looking in the wrong place. Robin2's code won't be impacted if you increase the time between transmissions. If yours still is, then something else in your setup is causing a problem.

It looked like some of the errors were related to the bigger integers (>100), so I decided to try splitting all of them down to 2 digit packets with limited success...

Forget about this. Totally complicates things unnecessarily.

This is running at 250000 baud.

A slower baud rate may have less chance of error.

What do you see at 115200?

I would work with the Serial monitor and not the oled, until you get the reliable data transmission sorted out. Display "flicker" should be a separate issue.

Ok I have found the source of the error. I cut the code right back to just the serial function and it was running stable. I then went back to the full code but switched different bits of it off until the issue disappeared, and the source of the error is:

  if (millis() - lastmillis >= 10) { 
    if (millis() - lastmillis >= 1000) { //Update every 1s
      SPEED_MPS = SPEED_MPH * 0.44704; //convert to distance meters per second
      distm = SPEED_MPS / 1 ; // spare conversion factor
      meterssubtotal += distm;  // add meters traveled to a total called meters
      updateodometer(); //add the meters traveled to the meters counter
      lastmillis = millis();
    }

    if (lastmillis > ShowServiceTime) { //Service timer display
      if (ShutDown != 1 && GearSetupMode != 1 && ChangeSource != 1) {
        checkdisplay(); // update lcd readings
      }
    }
  }

Specifically: checkdisplay();

This is the code the updates my OLED and it runs currently every 10ms as you see above. I tried switching it to 20ms and the problem didn’t go away.

I’ve attached the full sketch for information, but something to do with the 128x32 i2c display updating is causing misreads on the serial… Any ideas why??

RocketDash.ino (61.5 KB)

Specifically: checkdisplay();
This is the code the updates my OLED and it runs currently every 10ms as you see above. I tried switching it to 20ms and the problem didn't go away.

You are not the first person to encounter problems with the oled display effecting the sketch. My thinking is that the display.display() command which is being called every 10 ms is blocking and effecting the serial buffer.

What happens if you update the oled at the 1 second interval of the data update?

I can't look at your code on this tablet so here are a couple of theories.

You are sending the data frequently. All of it. CheckDisplay messes it up, so I suspect that the display functions are slow and over time, you fill the receive buffer and throw data away. Oled display libraries are often slow, usually because they keep a buffer that the display functions touch and then when commanded, every pixel is sent over the wire. So if you touch a single dot, you will send everything. That takes time.

Maybe the above is true but you're still fast enough to keep up. It could be that when you're thrashing i2c with all that data that you have crosstalk between it and your serial wires, causing corruption.

I suggest that you think about how often you need the data transmitted. I would first slow it down to something much slower just for testing. Maybe 500mS. There's no point sending it faster than you intend to use it which I think you were. Equally, you should consider how often the display can be updated and still be readable by a human.

I think that the different data items probably justify different display update rates and thus different transmission frequencies. Fuel state and RPM would seem to be poster children for different treatment. I liked your earlier approach of sending each data item in a separate packet for that reason.

Thanks very much for your insight chaps. I have altered every display function to check whether the key value to be displayed has changed or not, and only to update the OLED when new data is available. This has completely fixed the problem, but I will also look at the serial sending code to only send what is necessary like I used to have. It's looking nice and solid now which is a joy to see - thanks chaps, I think that was the last big hurdle to cross after a very long process! Now on to the long tail of tidying, tweaks and minor bug fixes :slight_smile:

Ok I have another question... I'm trying to convert robin's example 5 to be able to read varying data packets. To do this, I need to know if the relevant data is in the string like I did previously with:

if (tempChars.indexOf("S") > 0) {

I'm finding though that this generates an error as Robin's code doesn't use a string... Error:

request for member 'indexOf' in 'tempChars', which is of non-class type 'char [32]'

Is there an equivalent method of testing the array for presence of "S" without altering Robin's code?

Ultimately, I want to do something like this:

 if (tempChars.indexOf("S") > 0) {
       StrS1 = tempChars.indexOf('S');  //finds start location
       StrF1 = tempChars.lastIndexOf('S'); //finds finish location
       SPEED1 = tempChars.substring(StrS1 + 1, StrF1); //captures data String

   SPEED_MPH = SPEED1;
 }

strchr. It returns NULL if not found, else a pointer to the character.

I think you want the send the messages with a command letter and the value between the start and end markers.

For example


There are several ways to deal with the commands from here, but one simple method would be a switch case on the command letter which is received.

I think that parseData() could looks something like this

void parseData() {
  char command = tempChars[0];
  switch (command) {
    case 'S' :
      speed = atoi(tempChars[1]);
      break;
    case 'T':
      //temperature code
      break;
    case 'F':
      //fuel code
      break;
    default:
      Serial.println("not valid command");
  }

}

Thanks guys, I did a bit of reading and decided to change my packets to:

<1234S1234R123T123F>

This way I just need to find the final letter and split the string, so I’ve ended up with this:

  char * strtokIndx;

  if (strchr(tempChars, 'S') > 0) {
    strtokIndx = strtok(tempChars, 'S');
    SPEED_MPH = atoi(strtokIndx);

    if (strchr(strtokIndx, 'R') > 0) {
      strtokIndx = strtok(strtokIndx, 'R');
      RPM = atoi(strtokIndx);
    }

    if (strchr(strtokIndx, 'T') > 0) {
      strtokIndx = strtok(strtokIndx, 'T');
      COOLANT_TEMP = atoi(strtokIndx)/10;
    }

    if (strchr(strtokIndx, 'F') > 0) {
      strtokIndx = strtok(strtokIndx, 'F');
      FUEL_LEVEL = atoi(strtokIndx)/10;
    }
  }
  
  else if (strchr(tempChars, 'R') > 0) {
    strtokIndx = strtok(tempChars, 'R');
    RPM = atoi(strtokIndx);

    if (strchr(strtokIndx, 'T') > 0) {
      strtokIndx = strtok(strtokIndx, 'T');
      COOLANT_TEMP = atoi(strtokIndx)/10;
    }

    if (strchr(strtokIndx, 'F') > 0) {
      strtokIndx = strtok(strtokIndx, 'F');
      FUEL_LEVEL = atoi(strtokIndx)/10;
    }
  }
  else if (strchr(tempChars, 'T') > 0) {
    strtokIndx = strtok(tempChars, 'T');
    COOLANT_TEMP = atoi(strtokIndx)/10;

    if (strchr(strtokIndx, 'F') > 0) {
      strtokIndx = strtok(strtokIndx, 'F');
      FUEL_LEVEL = atoi(strtokIndx)/10;
    }
  }
  else if (strchr(tempChars, 'F') > 0) {
    strtokIndx = strtok(tempChars, 'F');
    FUEL_LEVEL = atoi(strtokIndx)/10;
  }

Which works pretty well but I’m still getting some misreads on just the Fuel and Temperature. I’ve put an exponential filter on the sending side so the signal is rock solid so it’s definitely the receiving end. I think I just need to add a pass filter to filter out noise at this point…

You know the sequence in which you send the parameters, so why bother with letters indicating them. Just use commas and you simply split on them (as in Robin’s examples).

Because I don’t always send all of them, I only send the values that have changed, though they will always be in this order so I need to check which ones are present.

Sill need to figure out a nice way of cleaning up the last two values though… My crude first attempt is this, which kind of works but doesn’t seem very elegant…

  if ((FUEL_LEVEL1 / 10) - FUEL_LEVEL < 50 ) {
    FUEL_LEVEL2 = FUEL_LEVEL1 / 10;
  }
  FUEL_LEVEL += 0.1 * (FUEL_LEVEL2 - FUEL_LEVEL);

  if ((COOLANT_TEMP1 / 10) - COOLANT_TEMP < 20 ) {
    COOLANT_TEMP2 = COOLANT_TEMP1 / 10;
  }
  COOLANT_TEMP += 0.1 * (COOLANT_TEMP2 - COOLANT_TEMP);

Fair enough.

A little example code; you can modify the receivedChars for testing.

char receivedChars[32] = "S1234,R4567,T89,F0";

void setup()
{
  int speed, revs, temp, fuel;

  Serial.begin(57600);
  while (!Serial);  // I'm using a Leonardo


  char *ptr;

  ptr = strtok(receivedChars, ",");

  if (ptr == NULL)
  {
    Serial.println("No comma found");
  }
  else
  {
    do
    {
      switch (ptr[0])
      {
        case 'S':
          speed = atoi(&ptr[1]);
          Serial.print("speed = "); Serial.println(speed);
          break;
        case 'R':
          revs = atoi(&ptr[1]);
          Serial.print("revs = "); Serial.println(revs);
          break;
        case 'T':
          temp = atoi(&ptr[1]);
          Serial.print("temparture = "); Serial.println(temp);
          break;
        case 'F':
          fuel = atoi(&ptr[1]);
          Serial.print("fuel = "); Serial.println(fuel);

          break;
      }
      ptr = strtok(NULL, ",");


    } while (ptr != NULL);
  }
}

void loop()
{
  // put your main code here, to run repeatedly:

}

The after a strtok, you have a character array; element 0 contains the 'type'.
&ptr[1] is the address of the array from element 1; that is the value

Note:
This will destroy the text in receivedChars which is usually not an issue. I have replaced strtokIndx by ptr.