Help with Best practice for sending multiple serial strings constantly

Projects Programming
1

Hi all,

I have a project sending multiple serial updates to a display constantly throughout the run time of the system.

From Sensor values and sensor pressures to brightness settings for touchpad and button logos etc.

Im coming across the issue now when if i include these serial writes to the display then it is affecting the rest of the loop with serial input coming to the arduino mega.

sometimes it will work sometimes it wont i'm assuming its because the buffer is filling up with outputs to send and cant process the inputs straight away or something along those lines.

i was wondering what is best practice to cope with this?

The following is some short code as a function sending data to the touchpad.

Each one of these if i comment out the others works flawlessly on its own but if i start to add all of them together then it progressively gets worse.

I currently have a clock in the background sending this data every 300ms
where the TPClockDelay = 300ms

void TpClock() {
  CurrentTime = millis();
  if (CurrentTime - TpClockDelay >= TimeLastTpDataSent) {
    SendTpData();
    TimeLastTpDataSent = millis();
  }
  }

so i started to only send certain data in certain run through of the loop example below. This makes it run smoother again and not fault when the mega is receiving commands but before i continue i didnt know if there is another way i should be going

void SendTpData() {
  if(DataPacket < 10){
    TankDataToSend();
  }
  if(10 < DataPacket < 20){
    HandbrakeDataToSend();
  }
  if(20 < DataPacket < 30){
    HeightSensorDataToSend();
  }
  if(30 < DataPacket < 40){ 
    DimmingData(); 
  }
  DataPacket++;
  if(DataPacket > 40){
    DataPacket = 0;
  }
}

Would this be the best way to cope with this or do you have other suggestions?

A few of these functions only need to be sent when changing and changing won't happen much ( as in LCD brightness, or if the handbrake has been applied or not, or if the headlights have been turned on etc) so these i could easily only write out when a change has occurred but the other functions such as pressures and height sensors will be constantly updating and would want to be updated to the touchpad display every 300ms or so.

Just after advise what would be recommended to do to send these functions without filling up buffers or trying to send too much data at once

thanks for your time

2

That is the wrong way to write those if statements. Should be:
if(DataPacket > 10 && DataPacket < 20)

if(10 < DataPacket < 20){ Will be evaluated as 10 < DataPacket = 1 or 0 then 1 or 0 < 20 which will always be true.

1 Like
3

My mistake,

Thanks for clarifying that!
Ill fix that now but as far as structuring or only sending serial data in different sections would this be the best way?

Each individual serial write is 8 bytes of data and as an idea theres 4x height sensors, 5 pressure sensors, 6 different input conditions that change logo visuals and multiple levels of info so for example the tank pressure has a mapped pressure reading but also a logo changes so another value mapped from 1 - 10 and a different icon based on what percentage the pressure is and all of these are in the format of:
as an example:
TP_TankPDisplay[8]{ 0x5A, 0XA5, 0X05, 0X82, 0X20, 0X05, 0X00, 0X00 };

So a lot of information being sent constantly then inputs are the same coming in that its processing

Would you say that doing it this way would be best practice? Or what would you recommend to send this much data in a way that wont overflow or upset things

4

If the units are seperated, this would be a great fit for CAN as it works great with 8 byte packets. Also you can prioritize the data. You might consider not updating the display so much, I do not feel most people will follow the data as fast as you are sending it. If you can slow that down your code becomes more forgiving.

5

The display has needles that move as the height changes anything over 200ms delay looks funny and laggy on the needles, while its not necessary to have them update that quick it does definitely look better. but running say height sensors on their own the code works fine and everything works but adding the others in its slowly becomes worse and worse the more data i include.

Not sure if sending at different times is the best way to do it perhaps maybe have a data block a clock B clock c clock and so on and have a 80 ms delay between those so it only sends 1 data block at a time every 80 ms or something like that so for example data a sends then 80ms later sends b then 80 ms later sends c and 80 ms later sends d so then back at a again after 240ms or so.

Not sure whats the best way to do it really

6

That is actually a common mistake. I suggest that you go, in the IDE, to File, Preferences and set compiler warnings to all. That line of code will generate a warning. Warnings will allow the code to compile, but not necessarily work right. You should always fix warnings, but you can't if you don't know about them.

What is sending data and to where? The serial transmit buffer is 64 bytes so as long as you send data fast enough to not fill that buffer serial output will not be effected.

I can't really say about best practice, I am far from a pro. For me the best practice is one that works and that I understand. Nothing wrong with any of those strategies. I have used most at different times.

For some data it makes sense to send periodically. Like regularly sampled ADC data. Send a data set each sample period.

Other times only when the data changes. Like data for an LCD.

The serial input basics tutorial may have information of interest.

7

If it were me I would update when something changes otherwise let it sit but every so often update everything say every 30 seconds. It will appear to be in real time but you will know the secret.

8

The mega is sending serial data from serial 1 to a Dwin display which has variables based on 8 bytes of data so for example one of the outputs being sent is the tank pressure

With the original definition of tank pressure aray being:
TP_TankPDisplay[8]{ 0x5A, 0XA5, 0X05, 0X82, 0X20, 0X05, 0X00, 0X00 };
0X5A 0XA5 is the header
0X05 means 5 more bytes of data to be received
0X82 means it is writing the value to dwin display ( or dwin is to read it )
0X20 is the variable high byte
0x05 is variable low byte
and the final 2 are the high byte and low byte data to be send

TankPressure = map(analogRead(TankSensor_Pin), 99, 918, 0, 200);

TP_TankPressure[7] = TankPressure;
    Serial1.write(TP_TankPressure, 8);

So you can see here the tank pressure value is stored into the 8th byte in the array ( from zero indexing) and then the string is written and sent to the dwin display which then updates the value on the display and changes a needle or value on the display etc.

9

I will be implementing this but for the likes of height sensor or pressure sensor data they will constantly change as they will slightly fluctuate always unless i put in some form of tolerance on the buffer out for visuals but the actual program is reading and using actual values which is also a possibility

10

Hint: these things will change faster then the brain can assimilate what the eye is seeing, use that to your advantage. You can also integrate the readings which is a normal practice.

11

The situation is not yet completely clear:

What is the maximum number of bytes that you send?

What is the maximum number of bytes that you receive?

What is the mimimum time between sending data?

What is the minimum time between receiving two packets of data?

At what baudrate are you sending/receiving this data?

depending on what these numbers really are,
depending on how much RAM ist left free

very different solutions can be found.

One possability is to increase the receive / send buffer
Without seeing your complete sketch and what all the rest of your code is doing

It is not possible to make good suggestions

best regards Stefan

12

Thanks for the reply:
I dont want to attach my whole sketch as its over 1000 lines of code and functions and also i have worked many hours on it haha.

But ill answer what i can.
Each data line sends 8 bytes for most other than page transitions which requires 9 bytes example 1 variable to the display is:
TP_TankPDisplay[8]{ 0x5A, 0XA5, 0X05, 0X82, 0X20, 0X05, 0X00, 0X00 };

The baud rate is 115200

Maximum bytes that are received is 9 bytes of data example:
{0x5A, 0XA5, 0X06, 0X46, 0x10, 0X20, 0X05, 0X00, 0X01 };

At the moment minimum time ( other than this sketch is literally line after line) to send data so would send 1 value then next then next then next all in a line of code with no gaps between

Received data is only received on a button push
so when the user pushes a button

the issue im running into is i have a certain function that when held in gives a string of data of 8 bytes to the arduino and when released it sends another string this works fine under no data sending from the arduino but when i load it up with all serial outputs being sent 1 after the other every 200ms then sometiems you let go and it performs as expected and other times it doesnt see the button release and continues to output as if the button was held down.

This issue goes away when commenting out lines of the output data if that makes sense.

For example i have:

void SendTpData() {
    HandbrakeDataToSend();
    HeightSensorDataToSend();
    DimmingData(); 
}

which has these issues apparent but if i say comment out a few of these lines like this:

void SendTpData() {
    HandbrakeDataToSend();
    // HeightSensorDataToSend();
   // DimmingData(); 
}

it works flawlessly and same if i comment out the handbrake data to send insetad of height sensor etc.

the batch of data being sent in this function is as follows:

void HeightSensorDataToSend(){
      TP_HS_P1[7] = (CurrentHSV[0]);
      TP_HS_V1[7] = (HeightSensorVoltage[0]);
      Serial1.write(TP_HS_P1, 8);
      Serial1.write(TP_HS_V1, 8);
      TP_HS_P2[7] = (CurrentHSV[1]);
      TP_HS_V2[7] = (HeightSensorVoltage[1]);
      Serial1.write(TP_HS_P2, 8);
      Serial1.write(TP_HS_V2, 8);
      TP_HS_P3[7] = (CurrentHSV[2]);
      TP_HS_V3[7] = (HeightSensorVoltage[2]);
      Serial1.write(TP_HS_P3, 8);
      Serial1.write(TP_HS_V3, 8);
      TP_HS_P4[7] = (CurrentHSV[3]);
      TP_HS_V4[7] = (HeightSensorVoltage[3]);
      Serial1.write(TP_HS_P4, 8);
      Serial1.write(TP_HS_V4, 8);
    }

where each serial write of 8 bytes is the same as above with slight variations ( variable position and value) so one of those serial writes would be

{ 0x5A, 0XA5, 0X05, 0X82, 0X20, 0X05, 0X00, 0X01 };

Hope that makes sense

13

Because of my above issue of line after line with no delay or space in time, would this be where i would be best to between all of sending sequences make a delay of say 5ms and basically trigger a Sending data event that sends each output 1x1 with a 5ms delay between them until all data has been sent then when the 300ms comes around again trigger the even again and repeat?

14

You can check with Serial1.availableForWrite() whether the data can be sent without blocking. If not enough bytes are free in the buffer then check next loop() iteration again.

2 Likes
15

wow, how did i miss this hahaha so simple ill give that a go now.

this is what happens when you have stared at code for days on end hahaha

thankyou this will surely help with some of the receiving side of things

16

I still think i have issues with sending data that close together though would it be a good idea to try and separate the sending data?

Or would you just say create a boolean variable for each set of data being sent say A data sent have them all false to start with and it goes through if it can write it then set the a data to true then on to b and same again test if can be written and if so write and set data b to true and so on and then that way at the very end have a check that needs all of them to be true before it sets them all back to false again and resends?

17

I'm actually thinking about essentially the same problem on an ESP32. How would I organize the display on a cockpit so that analog values (bars, needles...) update as fast as possible but values of lower priorities don't get lost.

If your data sets have equal priority but different frequency then you have a problem with your model. Should the slowest input really block all other transmissions? In this case I'd sort the sets by last transmission time and transmit the set with the longest time gap as soon as new data are available.

After all I think that you should proceed and write a first working program. Then find out where improvements really are needed.

18

It appears i have it working now hopefully no issues come up later but so far i did this:

void SendTpData() {
  if(Serial1.availableForWrite() >= 16 && DataSent == 0){
  TankDataToSend();
  D_println();
  D_print("Tank Data Sent");
  DataSent = 1;
  }
  if(Serial1.availableForWrite() >= 8 && DataSent == 1){
  HandbrakeDataToSend();
  D_println();
  D_print("Handbrake Data Sent");
  DataSent = 2;
  }
  if(Serial1.availableForWrite() >= 60 && DataSent == 2){
  HeightSensorDataToSend();
  D_println();
  D_print("Height Sens Data Sent");
  DataSent = 3;
  }
  if(Serial1.availableForWrite() >= 60 && DataSent == 3){
  HeightSensorVoltageToSend();
  D_println();
  D_print("Height Voltage Data Sent");
  DataSent = 4;
  }
  if(Serial1.availableForWrite() >= 16 && DataSent == 4){
  DimmingData(); 
  D_println();
  D_print("Dimming Data Sent");
  DataSent = 0;
  TimeLastTpDataSent = millis();
  }

so essentially check if the number of bytes that the function requires free to be able to be written is available and if so then write it and set the data sent value up by 1 then the next data to be sent checks and if not available then next loop it will be.

not sure if this is the correct way to be using the Serial1.availableForWrite() function as i'm not familiar with it but it seems to be working?

19

This is your choice. You miss the opportunity for members of this community to help you refine your code; no guarantees, but it is very likely a serious miss. OTOH, you avoid the distraction/discomfort of some serious headbanging, because it's often the case that you'll feel silly at the end of it, because your code wasn't 'up to snuff'.

As for your question, I consider the problem in terms of payload vs overhead. You've got 3/4 of the data you're sending committed to just introducing each payload. Pretty poor ratio.
You seem to have a fixed number of variables, and wish to send them all repetitively. In that case, it's best to just send "preamble, payload, terminator", where the payload is one complete data set, then shut up until the next interval.
If you MUST send each variable individually, then I would seriously consider each one in terms of how frequently it must be updated, on the basis of how much change needs to be displayed. Even pre-filtering the data to reduce display flicker could significantly reduce your data transmission. No one will ever notice that variable A gets updated 3x as often as variable B.
Barring all that, you may need to increase your baud rate* to ensure all the data goes promptly, and then fine-tune your receiving code to minimize delays at the receiving end. That's where this group could help you, if we had your code.

    • don't know, because you haven't even shared that!
1 Like
20

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.