Help with recreating Serial data

Hi all,

First of all please forgive the not so explanatory topic name and the very newbie questions that will follow, what I'm trying to do far exceeds my coding knowledge.

I have a LED controller(Pixelblaze) that connects to an external sensor board that adds some extra functionality(more analog pins, accelerometer, sound, light sensor) via an RX pin. For a project I'm currently making(a LED controller for my kids), I need more analog pins than the board can provide. As I don't need the sensor board itself, I want to use an Arduino in its stead and take advantage of the extra analog pins. Pixelblaze expects 5 analog pins coming from the sensor board, but I can send the data of the rest in the bytes used for the light sensor or accelerometer.

The sensor board is open source so there is enough documentation available, but given my limited understanding I don't know how to replicate the stream.

From the board's GitHub page:

The protocol is fairly simple:

  1. Each frame starts with "SB1.0" including a null character (6 bytes).
  2. The frequency information follows, as 32 x 16-bit unsigned integers.
  3. Then is the audio energy average, max frequency magnitiude, max frequency Hz, all 3 as 16-bit unsigned ints.
  4. Next the accelerometer information as 3 x 16-bit signed integers.
  5. The data from the Light sensor is next, as a single 16-bit unsigned integer.
  6. Followed by the 5 x 16-bit analog inputs (12-bit resolution, shifted up to 16 bits)
  7. Finally "END" including a null character (4 bytes).

If I understand this correctly, the total byte count for the above is 98. I tried using code from similar topics to read the stream, like Robin2's example from this thread, but I don't understand how to read(or recreate for that matter) data that has a string in the header(SB1.0) or how to place that in the startMarker bit of the above example, so all I get is endless rows that always start with different values.

I converted SB1.0 to bytes and did this:

const byte startMarker = (0x53, 0x42, 0x31, 0x2E, 0x30);

but I get nothing back in Serial Monitor.

The sensor board has an Arduino library that I guess makes it even easier for someone who understands, but not me...

//
// Created by Ben Hencke on 2019-07-28.
//

#ifndef SB_ARDUINO_PIXELBLAZESENSORBOARD_H
#define SB_ARDUINO_PIXELBLAZESENSORBOARD_H

#include <Arduino.h>


#ifndef ICACHE_RAM_ATTR
#define ICACHE_RAM_ATTR
#endif

typedef struct {
//    char header[6]; // "SB1.0\0"
    uint16_t frequencyData[32];
    uint16_t energyAverage;
    uint16_t maxFrequencyMagnitude;
    uint16_t maxFrequency; //in hz
    int16_t accelerometer[3];
    uint16_t light;
    uint16_t analogInputs[5];
//    char end[4]; // "END\0"
} __attribute__((__packed__)) SB10Frame;


class PixelblazeSensorBoard {
public:
    Stream &source;
    enum {HEADER, DATA, END} mode = HEADER;
    uint8_t pos = 0;
    SB10Frame sb10Frame;
    uint8_t frameBuffer[sizeof(SB10Frame)];
    unsigned long lastFrameTime = 0;

    PixelblazeSensorBoard(Stream &source) : source(source) {
    }

    uint16_t frequencyData(int index) {
        if (index < 0 || index > 31)
            return 0;
        return sb10Frame.frequencyData[index];
    }
    uint16_t energyAverage() {
        return sb10Frame.energyAverage;
    }
    uint16_t maxFrequencyMagnitude() {
        return sb10Frame.maxFrequencyMagnitude;
    }
    uint16_t maxFrequency() {
        return sb10Frame.maxFrequency;
    }
    int16_t accelerometerX() {
        return sb10Frame.accelerometer[1];
    }
    int16_t accelerometerY() {
        return sb10Frame.accelerometer[0];
    }
    int16_t accelerometerZ() {
        return sb10Frame.accelerometer[2];
    }
    uint16_t light() {
        return sb10Frame.light;
    }
    uint16_t analogInput(int index) {
        if (index < 0 || index > 4)
            return 0;
        return sb10Frame.analogInputs[index];
    }
    unsigned long dataAgeMillis() {
        return millis() - lastFrameTime;
    }

    bool readNextFrame() {
        if (source.available() < 1)
            return false;
        unsigned long timeout = 15;
        unsigned long startMs = millis();
        do {
            int c = source.read();
            if (c < 0)
                continue;
            if (processFrameByte(c))
                return true;
        } while (millis() - startMs < timeout);
        //in event of timeout reset state to sync back to header
        mode = HEADER;
        pos = 0;
        return false;
    }

    void ICACHE_RAM_ATTR readAvailable() {
        int c;
        while ((c = source.read()) >= 0) {
            processFrameByte(c);
        }
    }

    bool ICACHE_RAM_ATTR processFrameByte(uint8_t c) {
        bool result = false;
        switch (mode) {
            case HEADER:
                if (c != "SB1.0"[pos++]) //sync to header
                    pos = 0;
                if (pos == 6) { //full header found, including null byte
                    mode = DATA;
                    pos = 0;
                }
                break;
            case DATA:
                frameBuffer[pos++] = c;
                if (pos == sizeof(SB10Frame)) {
                    mode = END;
                    pos = 0;
                }
                break;
            case END:
                if (c != "END"[pos++]) {//sync to header
                    //lost sync somewhere in the middle, throw away the frame
                    mode = HEADER;
                    pos = 0;
                }
                if (pos == 4) { //full footer found, including null byte
                    mode = HEADER;
                    pos = 0;
                    memcpy(&sb10Frame, frameBuffer, sizeof(SB10Frame));
                    lastFrameTime = millis();
                    result = true;
                }
        }
        return result;
    }
};

#endif //SB_ARDUINO_PIXELBLAZESENSORBOARD_H

I already know how to read values from the analog inputs and setup my sketch for that. What I don't know is how to send the values in the above way.

With my limited understanding, I'd say that I have to do the following:

  1. Setup an array with the different bytes. This, I'm guessing will be in HEX, correct?
  2. Assign the input values to some of the bytes, say the light sensor and accelerometer, plus the 5 analog inputs already there.
  3. Send it via Serial at the interval Pixelblaze expects(40ms).

If someone could help me understand a bit, I would appreciate it.

This is the sketch I'm playing with at the moment but get no results in the Serial Monitor:

// Example 6 - Receiving binary data

#include <AltSoftSerial.h>
AltSoftSerial altSerial;
const int rxpin = 8;

const byte numBytes = 98;
byte receivedBytes[numBytes];
byte numReceived = 0;

boolean newData = false;

void setup() {
    Serial.begin(115200);
    altSerial.begin(115200);
    Serial.println("<Arduino is ready>");
}

void loop() {
    recvBytesWithStartEndMarkers();
    showNewData();
}

void recvBytesWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    byte startMarker = (0x53, 0x42, 0x31, 0x2E, 0x30, 0x00);
    byte endMarker = (0x45, 0x4E, 0x44, 0x00);
    byte rb;
   

    while (altSerial.available() > 0 && newData == false) {
        rb = altSerial.read();

        if (recvInProgress == true) {
            if (rb != endMarker) {
                receivedBytes[ndx] = rb;
                ndx++;
                if (ndx >= numBytes) {
                    ndx = numBytes - 1;
                }
            }
            else {
                receivedBytes[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                numReceived = ndx;  // save the number for use when printing
                ndx = 0;
                newData = true;
            }
        }

        else if (rb == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in (HEX values)... ");
        for (byte n = 0; n < numReceived; n++) {
            Serial.print(receivedBytes[n], HEX);
            Serial.print(' ');
        }
        Serial.println();
        newData = false;
    }
}

These need to be defined as arrays... currently they are defined as a single byte

    byte startMarker[] = {0x53, 0x42, 0x31, 0x2E, 0x30, 0x00};
    byte endMarker[] = {0x45, 0x4E, 0x44, 0x00};

... and when checking for the start/end markers you need to be comparing multiple bytes.

Could you maybe elaborate a bit?

I tried this:

 if (rb != endMarker[]) {

but you clearly meant something different...

rb is a single byte (the one just read).
endMarker[] is an array of 4 bytes {0x45, 0x4E, 0x44, 0x00}

you need to compare 4 bytes read with the 4 bytes in endMarker.

Given your data is fixed length (always 98 bytes) you might be better just looking for the start marker, then read in up to 98 bytes... then process.

To find the start marker you need to check 6 consecutive bytes read with each byte in the marker.

I'm sorry but I still don't know how to do this. I tried to find similar cases but couldn't.

Ok, maybe I'm doing something wrong in my conversions, but if I use this sketch to get a raw input:

#include <AltSoftSerial.h>
AltSoftSerial altSerial;
const int rxpin = 8;
byte c;
void setup() 
{
  Serial.begin(115200); // 9600 baud for the built-in serial port
  altSerial.begin(115200); // initialize the port, most GPS devices use 4800 baud
}
 


void loop() 
{
  if (altSerial.available() > 0) // any character arrived yet?
  {
    c = altSerial.read();   // if so, read it from the GPS
    Serial.print(c, HEX);        // and echo it to the serial console
    
  }
}

This is what I get(part of it anyway):

1E66800E0F886180600078600607807860000600600006006006078600607860060060060786078600E006078600607860786078006078607860780E080786078180FE086060078007E980FE0FE0FE0FE668698E001E6680F818FE0180000060060060060000006006078600600600607860786000006078607860780786078607807860786078078607860780786078600E0786060000F808607E0078001E66FE0FE0FE0FE6686986001E66800E0F88618000607860786000078607860060000600600006078600078600607860060786078607800607860786078078607860780786078607809E078601E00FE01801E0078007E980FE0FE0FE0FE668698E001E6680FE188018786000060060786078006006000786006006006006006006006006006078600600600600600607860786078006078600607860E001E187E180FE0180600600018660FE0FE0FE0FE668698E001E66800E0F88618786000060786000078600078600600607860786000078607860780060786078600E078607860780786078607807860786078078607860780E08078609800FE086060078007E980FE0FE0FE0FE6686986001E66800E0F8861806006078600600E0786006006006006000060786078607807860060060060786078600E0060786078607807860786078078607860780E0807860FE000FE08607E007801E66FE0FE0FE0FE6686986001E66800E018861

Using an online converter, SB1.0 equals 0x53, 0x42, 0x31, 0x2E, 0x30, or 5342312E30, and this string is nowhere in there. Any idea what I'm doing wrong?

Can you print a space between each character... might help to confirm what part of the record we are looking at.

Problem could well be with the baud rate when using AltSerial... this is likely not able to handle this speed.

Can you connect the Tx of the device to the Rx of the Arduino.

The following is an example of looking for start bytes (in this case 'ABC') read from the serial monitor, then reading a fixed length record. Data will be displayed once the total number of bytes (10) is read.

#define START_BYTES 3
#define LENGTH      10

const byte startMarker[START_BYTES] = {'A', 'B', 'C'};

byte receivedBytes[LENGTH];
uint8_t idx = 0;

boolean processData = false;
boolean started     = false;

void setup()
{
  Serial.begin(115200);
}

void loop()
{

  while (Serial.available() > 0)                           // Something to receive?
  {
    receivedBytes[idx++] = Serial.read();                  // Read next byte into array

    if (! started)                                         // Are we still trying to find the start marker?
    {
      if (receivedBytes[idx - 1] != startMarker[idx - 1])  // Check if the byte received matches expected.   
        idx = 0;                                           // If not, start over.

      if (idx == START_BYTES)                              // Have we got all 6 bytes?
        started = true;                                    // Then we're started, so will read reset of data.
    } 
    else                                                   // We're started.
    { 
      if (idx == LENGTH)                                   // Read to the end?
        processData = true;                                // Set the flag so the data gets processed.
    }
  }


  if (processData)                                         // Flag set?
  {
    for (uint8_t x = 0; x < LENGTH; x++)                   // Loop through the array.
    {
      Serial.println(receivedBytes[x], HEX);               // Display a character.
    }
    started = false;                                       // Start looking for the start marker again.
    processData = false;                                   // Clear the processing flag.
    idx = 0;                                               // Reset the array index to start over.
  }
}

I was using a pro mini clone and it turned out the problem was there. It's a 3.3v 8mhz variant and from what I gather it cannot handle such baud rates regardless if soft or normal serial is used. I switched to a Leonardo and got correct results.

Your code was very helpful, I replaced the start bytes and got a correct sequence, I think I know how to move on from here.

Thanks a lot for the time and effort you put in helping me!

Here's where I'm stuck at the moment:

I've created a new array(broken into parts for easier manipulation), and sending it via Serial1 TX every 40ms to a LED controller. I have also connected in Serial1 RX a sensor board that transmits the same byte array, again every 40ms. If I want to use some of the data of the sensor board, I do the following:

If a digital input becomes LOW, it invokes SensorBoard() (at the end of the code) that starts reading the incoming RX bytes, and maps some of them to the array.

For some reason the speed of the transmission is extremely slow when the values come from the sensor board(once every few seconds), if the input goes high it goes back to normal. I went through the code line by line(it's the one you provided in your reply) and couldn't find anything that could cause it- though I did change the while into if in the beginning as I read it would prevent any code outside the loop from continuing.

As it is a lot of data coming in, is it overflowing the buffer(sorry if this a stupid question)?

Any idea why?

#include <EEPROM.h>

#define EEPROM_SIZE 1

#define RXD2 16   // Serial1 Pins
#define TXD2 17


#include <Wire.h>
#include <Adafruit_INA219.h>

Adafruit_INA219 ina219_A;
//Adafruit_INA219 ina219_B(0x41);

//************ SENSOR BOARD START *******************

#define START_BYTES 5
#define LENGTH      98

const byte startMarker[START_BYTES] = {'S', 'B', '1', '.', '0'};

byte receivedBytes[LENGTH];
uint8_t idx = 0;

boolean processData = false;
boolean started     = false;

//************  SENSOR BOARD END  *******************

uint16_t Leds1 = 10;
uint16_t Leds2 = 10;

#include <ezButton.h>

#define DEBOUNCE_TIME 50 // the debounce time in millisecond, increase this time if it still chatters

ezButton button(5); // create ezButton object that attach to pin GIOP21



int SerialPeriod = 40;  // Serial1 Write interval
unsigned long SerialTime_now = 0;

int inaPeriod = 4000;     //INA219 Read Interval
unsigned long inaTime_now = 0;

int LedPeriod = 500;     // LED Count Interval
unsigned long LedTime_now;
bool ItWasDone = false;



uint16_t Colour1A  = 0;    // COLOURS!!!
uint16_t Colour1B  = 0;
uint16_t Colour1C  = 0;
uint16_t Colour1D  = 0;
uint16_t Colour2A  = 0;
uint16_t Colour2B  = 0;
uint16_t Colour2C  = 0;
uint16_t Colour2D  = 0;


byte header[] = {0x53, 0x42, 0x31, 0x2E, 0x30, 0x00};  //HEADER   SB1.0

byte freq[] =     {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,     // FREQUENCY 1/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 2/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 3/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 4/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 5/6
                   0x00, 0x00, 0x00, 0x00
                  };                                                    // FREQUENCY 6/6

byte audio[] =    {0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};    // AUDIO ENERGY

byte acc[] =      {0x00, 0x00, 0x7F, 0xFF, 0x7F, 0xFF};    //ACCELEROMETER

byte light[] =    {0x00, 0x00};    // LIGHT SENSOR

byte analog[] =  {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};    //ANALOG INPUTS

byte end[] =      {0x45, 0x4E, 0x44, 0x00};    // END



//  ***************  POTENTIOMETERS START **********************

const int NINPUTS = 4;
const byte inputPin[NINPUTS] = { 27, 14, 12, 13 };
int reading[NINPUTS];
float smoothedReading[NINPUTS];

void takeReadings() {
  for (int i = 0; i < NINPUTS; i++) {
    reading[i] = analogRead(inputPin[i]);
    smoothedReading[i] = smoothedReading[i] * .85 + reading[i] * .15;
  }
}

// ****************** POTENTIOMETERS END ***********************



void setup() {

  setCpuFrequencyMhz(20);   // Drop consumption, save the planet:)

  Serial.begin(115200);
  Serial1.begin(115200, SERIAL_8N1, RXD2, TXD2);

  Wire.begin();

  ina219_A.begin();
  //ina219_B.begin();

  EEPROM.begin(EEPROM_SIZE);

  Leds1 = EEPROM.read(0);

  Serial.println(Leds1);

  pinMode(5, INPUT_PULLUP);    // LED COUNT BUTTON

  pinMode(18, INPUT_PULLUP);    // SENSOR BOARD SELECTOR
  button.setDebounceTime(DEBOUNCE_TIME);

  uint32_t currentFrequency;   //INA219

}

void loop() {

  takeReadings();     // ANALOG INPUT SMOOTHING


  float shuntvoltage_A = 0;
  float busvoltage_A = 0;
  float current_mA_A = 0;
  float loadvoltage_A = 0;
  float power_mW_A = 0;

  shuntvoltage_A = abs(ina219_A.getShuntVoltage_mV());
  busvoltage_A = ina219_A.getBusVoltage_V();
  current_mA_A = abs(ina219_A.getCurrent_mA());
  power_mW_A = ina219_A.getPower_mW();
  loadvoltage_A = busvoltage_A + (shuntvoltage_A / 1000);
  /*
    float shuntvoltage_B = 0;
    float busvoltage_B = 0;
    float current_mA_B = 0;
    float loadvoltage_B = 0;
    float power_mW_B = 0;

    shuntvoltage_B = abs(ina219_B.getShuntVoltage_mV());
    busvoltage_B = ina219_B.getBusVoltage_V();
    current_mA_B = abs(ina219_B.getCurrent_mA());
    power_mW_B = ina219_B.getPower_mW();
    loadvoltage_B = busvoltage_B + (shuntvoltage_B / 1000);
  */

  button.loop(); // MUST call the loop() function first

  if (button.isPressed()) {
    Leds1 = 1000;
    //Leds2 = 1000;
    //Serial.print("Leds1 = "); Serial.println(Leds1);
    ItWasDone = true;
    LedTime_now = millis();
  }
  if (ItWasDone && ( millis() - LedTime_now >= LedPeriod) )
  {
    Leds1 = power_mW_A / 120;
    Leds1 = (((2 * Leds1) + 5) / 10) * 5;

    //Leds2 = power_mW_B / 120;
    //Leds2 = (((2 * Leds2) + 5) / 10) * 5;
    //EEPROM.write(0, Leds1);
    //EEPROM.write(1, Leds2);
    //EEPROM.commit();
    //Serial.print("Leds1 = "); Serial.println(Leds1);
    ItWasDone = false;
  }

  if (digitalRead(18) == HIGH) {

    //************  ANALOG INPUT START  ****************

    uint16_t Pot1A = smoothedReading[0] * 16;
    freq[0] = (Pot1A & 0xFF);
    freq[1] = ((Pot1A >> 8) & 0xFF);

    uint16_t Pot1B = smoothedReading[1] * 16;
    freq[2] = (Pot1B & 0xFF);
    freq[3] = ((Pot1B >> 8) & 0xFF);

    uint16_t Pot2A = smoothedReading[2] * 16;
    freq[4] = (Pot2A & 0xFF);
    freq[5] = ((Pot2A >> 8) & 0xFF);

    uint16_t Pot2B = smoothedReading[3] * 16;
    freq[6] = (Pot2B & 0xFF);
    freq[7] = ((Pot2B >> 8) & 0xFF);

    Colour1A = (analogRead(36)) * 16;
    freq[8] = (Colour1A & 0xFF);
    freq[9] = ((Colour1A >> 8) & 0xFF);

    Colour1B = (analogRead(39)) * 16;
    freq[10] = (Colour1B & 0xFF);
    freq[11] = ((Colour1B >> 8) & 0xFF);

    Colour1C = (analogRead(34)) * 16;
    freq[12] = (Colour1C & 0xFF);
    freq[13] = ((Colour1C >> 8) & 0xFF);

    Colour1D = (analogRead(35)) * 16;
    freq[14] = (Colour1D & 0xFF);
    freq[15] = ((Colour1D >> 8) & 0xFF);

    Colour2A = (analogRead(32)) * 16;
    freq[16] = (Colour2A & 0xFF);
    freq[17] = ((Colour2A >> 8) & 0xFF);

    Colour2B = (analogRead(33)) * 16;
    freq[18] = (Colour2B & 0xFF);
    freq[19] = ((Colour2B >> 8) & 0xFF);

    Colour2C = (analogRead(25)) * 16;
    freq[20] = (Colour2C & 0xFF);
    freq[21] = ((Colour2C >> 8) & 0xFF);

    Colour2D = (analogRead(26)) * 16;
    freq[22] = (Colour2D & 0xFF);
    freq[23] = ((Colour2D >> 8) & 0xFF);

    //***************  ANALOG INPUT END  *****************

  } else {
    SensorBoard();
  }

  analog[6] = (Leds1 & 0xFF); // lower byte    ANALOG 3    LED OUTPUT 1 VALUE
  analog[7] = ((Leds1 >> 8) & 0xFF); // upper byte

  analog[8] = (Leds2 & 0xFF); // lower byte    ANALOG 4    LED OUTPUT 2 VALUE
  analog[9] = ((Leds2 >> 8) & 0xFF); // upper byte

  if ((unsigned long)(millis() - SerialTime_now) > SerialPeriod) {
    SerialTime_now = millis();

    Serial1.write(header, sizeof(header));
    Serial1.write(freq, sizeof(freq));
    Serial1.write(audio, sizeof(audio));
    Serial1.write(acc, sizeof(acc));
    Serial1.write(light, sizeof(light));
    Serial1.write(analog, sizeof(analog));
    Serial1.write(end, sizeof(end));
  }


  if ((unsigned long)(millis() - inaTime_now) > inaPeriod) {
    inaTime_now = millis();
    Serial.print("Shunt Voltage: "); Serial.print(shuntvoltage_A); Serial.println(" mV");
    Serial.print("Bus Voltage:   "); Serial.print(busvoltage_A); Serial.println(" V");
    Serial.print("Load Voltage:  "); Serial.print(loadvoltage_A); Serial.println(" V");
    Serial.print("Current:       "); Serial.print(current_mA_A); Serial.println(" mA");
    Serial.print("Power:         "); Serial.print(power_mW_A); Serial.println(" mW");
    Serial.println("");


  }


}


void SensorBoard()
{

  if (Serial1.available() > 0)                           // Something to receive?
  {
    receivedBytes[idx++] = Serial1.read();                  // Read next byte into array

    if (! started)                                         // Are we still trying to find the start marker?
    {
      if (receivedBytes[idx - 1] != startMarker[idx - 1])  // Check if the byte received matches expected.
        idx = 0;                                           // If not, start over.

      if (idx == START_BYTES)                              // Have we got all 6 bytes?
        started = true;                                    // Then we're started, so will read reset of data.
    }
    else                                                   // We're started.
    {
      if (idx == LENGTH)                                   // Read to the end?
        processData = true;                                // Set the flag so the data gets processed.
    }
  }


  if (processData)                                         // Flag set?
  {
    for (uint8_t x = 0; x < LENGTH; x++)                   // Loop through the array.
    {
      freq[0] = receivedBytes[6];
      freq[1] = receivedBytes[7];
      freq[2] = receivedBytes[8];
      freq[3] = receivedBytes[9];
      freq[4] = receivedBytes[10];
      freq[5] = receivedBytes[11];
      freq[6] = receivedBytes[12];
      freq[7] = receivedBytes[13];
      freq[8] = receivedBytes[14];
      freq[9] = receivedBytes[15];
      freq[10] = receivedBytes[16];
      freq[11] = receivedBytes[17];
      freq[12] = receivedBytes[18];
      freq[13] = receivedBytes[19];
      freq[14] = receivedBytes[20];
      freq[15] = receivedBytes[21];
      freq[16] = receivedBytes[22];
      freq[17] = receivedBytes[23];
      freq[18] = receivedBytes[24];
      freq[19] = receivedBytes[25];
      freq[20] = receivedBytes[26];
      freq[21] = receivedBytes[27];
      freq[22] = receivedBytes[28];
      freq[23] = receivedBytes[29];
      freq[24] = receivedBytes[30];
      freq[25] = receivedBytes[31];
      freq[26] = receivedBytes[32];
      freq[27] = receivedBytes[33];
      freq[28] = receivedBytes[34];
      freq[29] = receivedBytes[35];
      freq[30] = receivedBytes[36];
      freq[31] = receivedBytes[37];
      audio[0] = receivedBytes[38];
      audio[1] = receivedBytes[39];
      audio[2] = receivedBytes[40];
      audio[3] = receivedBytes[41];
      audio[4] = receivedBytes[42];
      audio[5] = receivedBytes[43];
    }
    started = false;                                       // Start looking for the start marker again.
    processData = false;                                   // Clear the processing flag.
    idx = 0;                                               // Reset the array index to start over.
  }
}

By removing the while loop you now only read a single byte every time SensorBoard() is called... so it will take 98 calls before you will get all the data.

The while loop will only continue when there is something to read... once the serial buffer is empty it will exit the loop and continue with the rest of the code. @ 115200 baud it will take around 8.5ms to read 98 bytes.

Yet when I put the while back in, the output was completely frozen...

If you send the data less often, say every 100ms, what happens?

I went all the way up to 300ms, no change, with while instead of if it's just stalled.

Can you post your latest code.

It's the same as before, only difference the while.

#include <EEPROM.h>

#define EEPROM_SIZE 1

#define RXD2 16   // Serial1 Pins
#define TXD2 17


#include <Wire.h>
#include <Adafruit_INA219.h>

Adafruit_INA219 ina219_A;
//Adafruit_INA219 ina219_B(0x41);

//************ SENSOR BOARD START *******************

#define START_BYTES 5
#define LENGTH      98

const byte startMarker[START_BYTES] = {'S', 'B', '1', '.', '0'};

byte receivedBytes[LENGTH];
uint8_t idx = 0;

boolean processData = false;
boolean started     = false;

//************  SENSOR BOARD END  *******************

uint16_t Leds1 = 10;
uint16_t Leds2 = 10;

#include <ezButton.h>

#define DEBOUNCE_TIME 50 // the debounce time in millisecond, increase this time if it still chatters

ezButton button(5); // create ezButton object that attach to pin GIOP21



int SerialPeriod = 40;  // Serial1 Write interval
unsigned long SerialTime_now = 0;

int inaPeriod = 4000;     //INA219 Read Interval
unsigned long inaTime_now = 0;

int LedPeriod = 500;     // LED Count Interval
unsigned long LedTime_now;
bool ItWasDone = false;



uint16_t Colour1A  = 0;    // COLOURS!!!
uint16_t Colour1B  = 0;
uint16_t Colour1C  = 0;
uint16_t Colour1D  = 0;
uint16_t Colour2A  = 0;
uint16_t Colour2B  = 0;
uint16_t Colour2C  = 0;
uint16_t Colour2D  = 0;


byte header[] = {0x53, 0x42, 0x31, 0x2E, 0x30, 0x00};  //HEADER   SB1.0

byte freq[] =     {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,     // FREQUENCY 1/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 2/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 3/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 4/6
                   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,    // FREQUENCY 5/6
                   0x00, 0x00, 0x00, 0x00
                  };                                                    // FREQUENCY 6/6

byte audio[] =    {0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};    // AUDIO ENERGY

byte acc[] =      {0x00, 0x00, 0x7F, 0xFF, 0x7F, 0xFF};    //ACCELEROMETER

byte light[] =    {0x00, 0x00};    // LIGHT SENSOR

byte analog[] =  {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};    //ANALOG INPUTS

byte end[] =      {0x45, 0x4E, 0x44, 0x00};    // END



//  ***************  POTENTIOMETERS START **********************

const int NINPUTS = 4;
const byte inputPin[NINPUTS] = { 27, 14, 12, 13 };
int reading[NINPUTS];
float smoothedReading[NINPUTS];

void takeReadings() {
  for (int i = 0; i < NINPUTS; i++) {
    reading[i] = analogRead(inputPin[i]);
    smoothedReading[i] = smoothedReading[i] * .85 + reading[i] * .15;
  }
}

// ****************** POTENTIOMETERS END ***********************



void setup() {

  setCpuFrequencyMhz(20);   // Drop consumption, save the planet:)

  Serial.begin(115200);
  Serial1.begin(115200, SERIAL_8N1, RXD2, TXD2);

  Wire.begin();

  ina219_A.begin();
  //ina219_B.begin();

  EEPROM.begin(EEPROM_SIZE);

  Leds1 = EEPROM.read(0);

  Serial.println(Leds1);

  pinMode(5, INPUT_PULLUP);    // LED COUNT BUTTON

  pinMode(18, INPUT_PULLUP);    // SENSOR BOARD SELECTOR
  button.setDebounceTime(DEBOUNCE_TIME);

  uint32_t currentFrequency;   //INA219

}

void loop() {

  takeReadings();     // ANALOG INPUT SMOOTHING


  float shuntvoltage_A = 0;
  float busvoltage_A = 0;
  float current_mA_A = 0;
  float loadvoltage_A = 0;
  float power_mW_A = 0;

  shuntvoltage_A = abs(ina219_A.getShuntVoltage_mV());
  busvoltage_A = ina219_A.getBusVoltage_V();
  current_mA_A = abs(ina219_A.getCurrent_mA());
  power_mW_A = ina219_A.getPower_mW();
  loadvoltage_A = busvoltage_A + (shuntvoltage_A / 1000);
  /*
    float shuntvoltage_B = 0;
    float busvoltage_B = 0;
    float current_mA_B = 0;
    float loadvoltage_B = 0;
    float power_mW_B = 0;

    shuntvoltage_B = abs(ina219_B.getShuntVoltage_mV());
    busvoltage_B = ina219_B.getBusVoltage_V();
    current_mA_B = abs(ina219_B.getCurrent_mA());
    power_mW_B = ina219_B.getPower_mW();
    loadvoltage_B = busvoltage_B + (shuntvoltage_B / 1000);
  */

  button.loop(); // MUST call the loop() function first

  if (button.isPressed()) {
    Leds1 = 1000;
    //Leds2 = 1000;
    //Serial.print("Leds1 = "); Serial.println(Leds1);
    ItWasDone = true;
    LedTime_now = millis();
  }
  if (ItWasDone && ( millis() - LedTime_now >= LedPeriod) )
  {
    Leds1 = power_mW_A / 120;
    Leds1 = (((2 * Leds1) + 5) / 10) * 5;

    //Leds2 = power_mW_B / 120;
    //Leds2 = (((2 * Leds2) + 5) / 10) * 5;
    //EEPROM.write(0, Leds1);
    //EEPROM.write(1, Leds2);
    //EEPROM.commit();
    //Serial.print("Leds1 = "); Serial.println(Leds1);
    ItWasDone = false;
  }

  if (digitalRead(18) == HIGH) {

    //************  ANALOG INPUT START  ****************

    uint16_t Pot1A = smoothedReading[0] * 16;
    freq[0] = (Pot1A & 0xFF);
    freq[1] = ((Pot1A >> 8) & 0xFF);

    uint16_t Pot1B = smoothedReading[1] * 16;
    freq[2] = (Pot1B & 0xFF);
    freq[3] = ((Pot1B >> 8) & 0xFF);

    uint16_t Pot2A = smoothedReading[2] * 16;
    freq[4] = (Pot2A & 0xFF);
    freq[5] = ((Pot2A >> 8) & 0xFF);

    uint16_t Pot2B = smoothedReading[3] * 16;
    freq[6] = (Pot2B & 0xFF);
    freq[7] = ((Pot2B >> 8) & 0xFF);

    Colour1A = (analogRead(36)) * 16;
    freq[8] = (Colour1A & 0xFF);
    freq[9] = ((Colour1A >> 8) & 0xFF);

    Colour1B = (analogRead(39)) * 16;
    freq[10] = (Colour1B & 0xFF);
    freq[11] = ((Colour1B >> 8) & 0xFF);

    Colour1C = (analogRead(34)) * 16;
    freq[12] = (Colour1C & 0xFF);
    freq[13] = ((Colour1C >> 8) & 0xFF);

    Colour1D = (analogRead(35)) * 16;
    freq[14] = (Colour1D & 0xFF);
    freq[15] = ((Colour1D >> 8) & 0xFF);

    Colour2A = (analogRead(32)) * 16;
    freq[16] = (Colour2A & 0xFF);
    freq[17] = ((Colour2A >> 8) & 0xFF);

    Colour2B = (analogRead(33)) * 16;
    freq[18] = (Colour2B & 0xFF);
    freq[19] = ((Colour2B >> 8) & 0xFF);

    Colour2C = (analogRead(25)) * 16;
    freq[20] = (Colour2C & 0xFF);
    freq[21] = ((Colour2C >> 8) & 0xFF);

    Colour2D = (analogRead(26)) * 16;
    freq[22] = (Colour2D & 0xFF);
    freq[23] = ((Colour2D >> 8) & 0xFF);

    //***************  ANALOG INPUT END  *****************

  } else {
    SensorBoard();
  }

  analog[6] = (Leds1 & 0xFF); // lower byte    ANALOG 3    LED OUTPUT 1 VALUE
  analog[7] = ((Leds1 >> 8) & 0xFF); // upper byte

  analog[8] = (Leds2 & 0xFF); // lower byte    ANALOG 4    LED OUTPUT 2 VALUE
  analog[9] = ((Leds2 >> 8) & 0xFF); // upper byte

  if ((unsigned long)(millis() - SerialTime_now) > SerialPeriod) {
    SerialTime_now = millis();

    Serial1.write(header, sizeof(header));
    Serial1.write(freq, sizeof(freq));
    Serial1.write(audio, sizeof(audio));
    Serial1.write(acc, sizeof(acc));
    Serial1.write(light, sizeof(light));
    Serial1.write(analog, sizeof(analog));
    Serial1.write(end, sizeof(end));
  }


  if ((unsigned long)(millis() - inaTime_now) > inaPeriod) {
    inaTime_now = millis();
    Serial.print("Shunt Voltage: "); Serial.print(shuntvoltage_A); Serial.println(" mV");
    Serial.print("Bus Voltage:   "); Serial.print(busvoltage_A); Serial.println(" V");
    Serial.print("Load Voltage:  "); Serial.print(loadvoltage_A); Serial.println(" V");
    Serial.print("Current:       "); Serial.print(current_mA_A); Serial.println(" mA");
    Serial.print("Power:         "); Serial.print(power_mW_A); Serial.println(" mW");
    Serial.println("");


  }


}


void SensorBoard()
{

  while (Serial1.available() > 0)                           // Something to receive?
  {
    receivedBytes[idx++] = Serial1.read();                  // Read next byte into array

    if (! started)                                         // Are we still trying to find the start marker?
    {
      if (receivedBytes[idx - 1] != startMarker[idx - 1])  // Check if the byte received matches expected.
        idx = 0;                                           // If not, start over.

      if (idx == START_BYTES)                              // Have we got all 6 bytes?
        started = true;                                    // Then we're started, so will read reset of data.
    }
    else                                                   // We're started.
    {
      if (idx == LENGTH)                                   // Read to the end?
        processData = true;                                // Set the flag so the data gets processed.
    }
  }


  if (processData)                                         // Flag set?
  {
    for (uint8_t x = 0; x < LENGTH; x++)                   // Loop through the array.
    {
      freq[0] = receivedBytes[6];
      freq[1] = receivedBytes[7];
      freq[2] = receivedBytes[8];
      freq[3] = receivedBytes[9];
      freq[4] = receivedBytes[10];
      freq[5] = receivedBytes[11];
      freq[6] = receivedBytes[12];
      freq[7] = receivedBytes[13];
      freq[8] = receivedBytes[14];
      freq[9] = receivedBytes[15];
      freq[10] = receivedBytes[16];
      freq[11] = receivedBytes[17];
      freq[12] = receivedBytes[18];
      freq[13] = receivedBytes[19];
      freq[14] = receivedBytes[20];
      freq[15] = receivedBytes[21];
      freq[16] = receivedBytes[22];
      freq[17] = receivedBytes[23];
      freq[18] = receivedBytes[24];
      freq[19] = receivedBytes[25];
      freq[20] = receivedBytes[26];
      freq[21] = receivedBytes[27];
      freq[22] = receivedBytes[28];
      freq[23] = receivedBytes[29];
      freq[24] = receivedBytes[30];
      freq[25] = receivedBytes[31];
      freq[26] = receivedBytes[32];
      freq[27] = receivedBytes[33];
      freq[28] = receivedBytes[34];
      freq[29] = receivedBytes[35];
      freq[30] = receivedBytes[36];
      freq[31] = receivedBytes[37];
      audio[0] = receivedBytes[38];
      audio[1] = receivedBytes[39];
      audio[2] = receivedBytes[40];
      audio[3] = receivedBytes[41];
      audio[4] = receivedBytes[42];
      audio[5] = receivedBytes[43];
    }
    started = false;                                       // Start looking for the start marker again.
    processData = false;                                   // Clear the processing flag.
    idx = 0;                                               // Reset the array index to start over.
  }
}

I'm thinking that I'm not doing any proper debugging at the moment, at the other end there is a microcontroller that expects the messages to come at a specific order, so if due to speed or whatever the header is lost it doesn't refresh anything. I'll print the array on serial monitor see if the refresh is fast.

Yes I think you really need to confirm exactly what you are receiving... it may not be what you expect.