Serial streaming and storing packets

Projects Programming
1

Greetings,

The goal is to log data from a Chargery Battery Management System (BMS).
http://www.chargery.com/doc/Chargery_BMS8T_specification_V3.0.pdf

I've converted the +5V/-5V serial signal out of the BMS to 5V that is read by a Mega pin 19 (RX1).

This is what the data looks like when Mega is reading the stream and then directly outputting to serial monitor.

// ChargerySniffer

 
  #define Serial1_baud 115200
  #define RX 1                // RX = pin 19 Mega
  const uint32_t DataDelay = 2000;  //Delay to determine when data stream ended

  uint32_t startMicros;                   
  static boolean busyFlag = false;
 
  
  void setup() 
  {
    Serial.begin(115200);
    Serial.println("Hey There");

    Serial1.begin (Serial1_baud);
  }
  void loop() 
  {
   
   if (Serial1.available() > 0) 
    {
      int inByte = Serial1.read();
      Serial.print(inByte, HEX);
      Serial.print(' ');
      busyFlag = true;
      startMicros = micros();
    }  
      
 //   else
    {
     if (busyFlag == true && micros() > (startMicros + DataDelay))
     {
       Serial.println(' ');
       busyFlag = false;
       
     }
    }
  }

Data:

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.04.09 08:39:22 =~=~=~=~=~=~=~=~=~=~=~=
Hey There
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 56 16 C F6 C FD C FE C F9 0 0 0 0 0 48 0 1E 5F 93  
68 3A 3A 33 D A  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 56 16 C F6 C FD C FE C FA 0 0 0 0 0 49 0 1E 5F 95  
24 24 57 F D 98 2 0 0 0 97 0 98 5F E3  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
68 3A 3A 33 D A  
24 24 56 16 C F6 C FD C FE C FA 0 0 0 0 0 49 0 1E 5F 95  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 56 16 C F6 C FC C FE C FA 0 0 0 0 0 46 0 1E 5F 91  
24 24 57 F D 98 2 0 0 0 97 0 98 5F E3  
68 3A 3A 33 D A  
24 24 57 F D 98 2 0 0 0 97 0 98 5F E3  
24 24 56 16 C F6 C FC C FE C F9 0 0 0 0 0 46 0 1E 5F 90  
24 24 57 F D 98 2 0 0 0 97 0 98 5F E3  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 56 16 C F6 C FC C FE C FA 0 0 0 0 0 46 0 1E 5F 91  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
68 3A 3A 33 D A  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 56 16 C F7 C FC C FE C F7 0 0 0 0 0 48 0 1E 5F 91  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 57 F D 98 2 0 0 0 97 0 97 5F E2  
24 24 56 16 C F6 C FE C FE C F9 0 0 0 0 0 46 0 1E 5F 92  
68 3A 3A 33 D A  
24 24 57 F D 98 2 0 0 0 97 0 98 5F E3  
24 24 57 F D 98 2 0 0 0 97 0 98 5F E3  
24 24 56 16 C F5 C FD C FE C F9 0 0 0 0 0 43 0 1E 5F 8D  
24 24 57 F D 98 2 0 0 0 96 0 98 5F E2

Data begins with a two byte header (0x24, 0x24) and then a 1 byte command (either 0x56 or 0x57), followed by data and finishing up with a check sum. I don't know what's going on with data packet: 68 3A 3A 33 D A . Not a concern.

I'm having problems storing the packets into an array.

My sketch:

// ChargeryArray2

const byte numBytes = 30;
int receivedData[numBytes];   // an array to store the received data

#define RX 1                // RX = pin 19 Mega

void setup()
{
  Serial.begin(115200);
  Serial.println("Hey There");

  Serial1.begin (115200);
}
void loop()
{

  getData56();

  delay(10);
  displayData();
  receivedData[1] = 0;

  getData57();
  delay(10);
  displayData();
  receivedData[1] = 0;
}


void getData56() {  // 0x56 = 86
  int inByte;
  int inByte0;
  int inByte1;
  if (Serial1.available() > 0)
  {


    do {

      inByte0 = Serial1.read();
      inByte1 = Serial1.read();
    } while (inByte0 != 36 && inByte1 != 86);   // header = 0x24 = 36

    for (int i = 2; i < 21; i++)
    {
      inByte = Serial1.read();
      receivedData[i] = inByte;

    }
    receivedData[0] = inByte0;
    receivedData[1] = inByte1;
  }

}
void getData57() {  // 0x57 = 87
  int inByte;
  int inByte0;
  int inByte1;
  if (Serial1.available() > 0)
  {
    do {

      inByte0 = Serial1.read();
      inByte1 = Serial1.read();
    } while (inByte0 != 36 && inByte1 != 87);   // header = 0x24

    for (int i = 2; i < 15; i++)
    {
      inByte = Serial1.read();
      receivedData[i] = inByte;

    }
    receivedData[0] = inByte0;
    receivedData[1] = inByte1;
  }

}


void displayData() {
  if (receivedData[1] == 86) {
    for (int i = 0; i < 21; i++) {
      Serial.print(receivedData[i], HEX);
      Serial.print(" ");
    }
    Serial.println();
  }

  if (receivedData[1] == 87) {
    for (int i = 0; i < 14; i++) {
      Serial.print(receivedData[i], HEX);
      Serial.print(" ");
    }
    Serial.println();
  }
}

Stored data:

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.04.09 09:16:05 =~=~=~=~=~=~=~=~=~=~=~=
Hey There
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
FFFFFFFF 57 FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 
24 57 F D 98 2 0 0 0 98 0 9A 5F E6 
24 57 F D 98 2 0 0 0 98 0 9A 5F E6 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 57 F D 98 2 0 0 0 99 0 9A 5F E7 
24 57 F D 98 2 0 0 0 99 0 9A 5F E7 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 56 16 C F9 C FC C FE C FC 0 0 0 0 0 5A 0 21 5F AD 
24 57 F D 98 2 0 0 0 98 0 9A 5F E6 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 56 16 C F9 C FC C FE C FD 0 0 0 0 0 58 0 24 5F AF 
24 56 16 C F9 C FD C FE C FC 0 0 0 0 0 5D 0 24 5F B4 
24 57 F D 98 2 0 0 0 99 0 9A 5F E7 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 98 0 9A 5F E6 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
FFFFFFFF 57 FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 56 16 C F9 C FE C FE C FD 0 0 0 0 FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 
24 57 F D 98 2 0 0 0 98 0 9A 5F E6 
24 57 F D 98 2 0 0 0 98 0 9A 5F E6 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 56 16 C F9 C FD C FD C FA 0 0 0 0 FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 98 0 99 5F E5 
24 57 F D 98 2 0 0 0 99 0 9A 5F E7 
24 57 F D 98 2 0 0 0 99 0 99 5F E6 
24 56 16 C F9 C FE C FD C FC 0 0 0 0 0 5B 0 21 5F AF

Sometimes the data flips out with a series of "FFFFFFFF" when displaying the 0x56 packet.

Thanks for any assistance.

bms24T_BMS16T_BMS8T_additional_protocol-RS232.pdf (154 KB)

2

In your first sketch whenever Serial1.available() > 0 you are only reading one byte which is OK because each time through loop() you read a byte until no more are available.

In your second sketch when Serial1.available() > 0 you are trying to read multiple bytes even though you can read faster than the bytes can be received. This is why you are getting FFs because there is no serial data available. You should only try to read the number of bytes of available or less. In between reading bytes you can loop while Serial1.available() == 0.

3

Thanks Todd. Would you mind showing where "while Serial1.available() == 0" needs to go.

Within the loop(), I inserted a 1 sec delay between getData56(); and getData57();. Though not perfect, it did clean up the data a bit.

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.04.09 14:09:55 =~=~=~=~=~=~=~=~=~=~=~=
Hey There
24 57 F D 98 2 0 0 0 9A 0 9A 5F E8 
24 57 F D 98 2 0 0 0 99 0 24 24 57 
24 56 16 C F9 C FC C FE C FD 0 0 0 0 0 5B 0 21 5F AF 
24 56 16 C F9 C FD C FE C FC 0 0 0 0 0 5E 0 21 5F B2 
24 56 16 C F9 C FD C FE C FC 0 0 0 0 0 5A 0 21 5F AE 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9A 5F E8 
24 57 F 68 3A 3A 33 D A 24 24 57 F D 
24 56 16 C F9 C FD C FE C FA 0 0 0 0 0 5B 0 22 5F AE 
24 56 16 C F9 C FE C FE C FA 0 0 0 0 0 5A 0 21 5F AD 
24 56 16 C F9 C FD C FE C FA 24 24 57 F 24 24 57 F D 98 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9A 5F E8 
24 56 16 C F9 C FC C FE C FD 0 0 0 0 24 57 F D 98 2 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 56 16 C F9 C FC C FE C FA 0 0 0 0 0 5B 0 24 5F AF 
24 56 16 C F9 C FC C FE C FD 0 0 0 0 0 5D 0 21 5F B1 
24 56 16 C F9 C FD C FE C FA 0 0 0 0 0 5D 0 21 5F AF 
24 56 16 C F9 C FE C FE C FC 0 0 0 0 24 24 57 F D 98 
24 56 16 C F9 24 24 57 F D 98 2 0 0 0 24 24 57 F D 98 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 56 16 C F9 C FD C FE C FD 24 24 57 F 3A 3A 33 D A 24 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9B 0 24 24 57 
24 56 16 C F9 C FC C FD C FA 0 0 0 0 0 58 0 22 5F A9 
24 56 16 C F9 C FD C FE C FC 0 0 0 0 0 5B 0 22 5F B0 
24 56 16 C F9 C FC C FE C FD 0 0 0 0 0 58 0 21 5F AC 
24 57 F D 24 24 57 F D 98 2 0 0 0 
0 57 0 21 5F 24 24 57 F D 98 2 0 0 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9B 0 9A 5F E9 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9B 0 24 24 57 
24 56 16 C F7 C FE C FB C FC 0 0 0 0 0 5B 0 22 5F AC 
24 57 F D 24 24 57 F D 98 2 0 0 0 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 56 16 C F9 C FC C FE C FA 0 0 0 0 0 5A 0 22 5F AC 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 56 16 C F9 C FD C FD C FD 0 0 0 0 24 57 F D 98 2 
24 56 16 C F9 C FE C FE C FA 0 0 0 0 24 24 57 F D 98 
24 56 16 C F9 C FE C FE C FD 0 0 0 0 24 24 57 F D 98 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 56 16 C F9 C FC C FE C FA 0 0 0 0 0 5A 0 21 5F AB 
24 56 16 C F9 C FE C FE C FC 0 0 0 0 0 5A 0 21 5F AF 
24 56 16 C F9 C FD C FE C FC 0 0 0 0 0 5B 0 21 5F FFFFFFFF 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9B 0 9B 5F EA 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 57 F D 98 2 0 0 0 9A 0 9B 5F E9 
24 56 16 C F9 C FC C FE C FA 0 0 0 0 0 5A 0 22 5F AC 
24 56 16 C F9 C FD C FB C FD 0 0 0 0 0 5A 0 21 5F AC 
24 56 16 C F9 C FC C FE C FD 0 0 0 0 0 5B 0 22 5F B0 
24 56 16 C F9 C FE C FE C FC 0 0 0 0 24 24 57 F D 98

All data packets should start with either 24 56 or 24 57. However, I notice this got through "0 57 0 21 5F 24 24 57 F D 98 2 0 0 "

4

I would add a simple function like this:

int readSerial1Byte()
{
  while (!Serial1.available());
  return Serial1.read();
}

Then everywhere you have a Serial1.read() replace it with a readSerial1Byte().

Also, you could easily restructure your getData56() and getData57() functions into one function. Take a look at your do-while loops. As soon as inByte0 is 36 it will exit the loop no matter the value of inByte1 since that will cause the loop condition to become false.

5

Gerry48:
All data packets should start with either 24 56 or 24 57. However, I notice this got through "0 57 0 21 5F 24 24 57 F D 98 2 0 0 "

Just noticed you said this. This is because of what I said previously.

"Take a look at your do-while loops. As soon as inByte0 is 36 it will exit the loop no matter the value of inByte1 since that will cause the loop condition to become false."

However it is also true that if inByte1 is 56 (or 57 for the other loop) it will exit the loop regardless the value of inByte0.

Your code needs to be restructured because it is not properly handling the 0x24 0x24 packet header. I will look at it some more in a while.

6

This code simplifies your header detection and also waits for serial data to arrive before attempting a read

// ChargeryArray2

const byte numBytes = 30;
int receivedData[numBytes];   // an array to store the received data

//#define RX 1                // RX = pin 19 Mega

void setup() {
  Serial.begin(115200);
  Serial.println("Hey There");
  Serial1.begin (115200);
}

void loop() {

  getData();
  delay(10);
  displayData();
  receivedData[1] = 0;
}

void displayData() {
  int packetSize = 0;

  if (receivedData[1] == 0x56) {
    packetSize = 15;
  }
  else if (receivedData[1] == 0x57 ) {
    packetSize = 21;
  }
  for (int i = 0; i < packetSize; i++) {
    if (receivedData[i] < 0x10) Serial.print( "0" );
    Serial.print(receivedData[i], HEX);
    Serial.print(" ");
  }
  Serial.println();
}


char readOneByte() {
  // wait for a byte to arrive
  while ( Serial1.available() < 1 ) {
    delay(2); // wait
  }
  return Serial1.read();
}

void getData() {
  // all packets start with 0x24, 0x56 or 0x24, 0x57
  int inByte;
  int packetSize;

  while (1) {
    while ( Serial.read() != 0x24 ) {
      delay(2); // wait for start of header
    }
    inByte = readOneByte();
    if ( inByte == 0x56 or inByte == 0x57 ) {
      break;
    }
    receivedData[0] = 0x24;
    receivedData[1] = inByte;

    if ( inByte == 0x56 ) {
      packetSize = 15;
    }
    else {
      packetSize = 21;
    }
    for (int i = 2; i < 21; i++) {
      receivedData[i] = readOneByte();
    }
  }
}
7

Here is restructured code which checks for invalid packet types and reads the number bytes based on the protocol spec you attached. It will also make sure you don't overrun the receive buffer and display the number of bytes discarded which may indicate errors in your packet processing. You could further refine the code by creating structures based on the packet types. Note that this code does not store start of packet bytes since they are not useful.

I know the code builds but I have not tested it. Study it and hopefully it will make sense.

// These can be modified to accommodate changes in packets without having
// to change processing code.
const byte STARTBYTE = '

;
const byte validPackets[] = {0x56, 0x57};
const int numPacketTypes = sizeof(validPackets)/sizeof(validPackets[0]);

// Receive data buffer
const int maxBytes = 30;
byte receivedData[maxBytes];  // an array to store the received data

void setup()
{
  Serial.begin(115200);
  Serial.println("ChargerySniffer");
  Serial1.begin (115200);
}

void loop()
{
  // Get a packet and display it if it is valid
  if (getPacket())
  {
    displayPacket();
  }
  else
  {
    Serial.print("Invalid packet type received: ");
    Serial.println(receivedData[0], HEX);
  }
}

byte readSerial1Byte()
{
  while (!Serial1.available());
  return Serial1.read();
}

bool getPacket()
{
  int discardCnt = 0;
 
  // Get start of packet
  while (true)
  {
    int startCnt = 0;

if (readSerial1Byte() == STARTBYTE)
    {
      // Increment start counter and see if this is the 2nd one
      startCnt++;
      if (startCnt == 2) break;
    }
    else
    {
      // Not a start byte so reset counter
      startCnt = 0;
      discardCnt++;
    }
  }

// If you are interested in how many bytes were discarded you can print or store
  // disacrdCnt variable here. 
  // The first call of this function may have disacrded bytes. Subsequent calls
  // with discarded bytes may indicate software or serial line errors.
  if (discardCnt > 0)
  {
    Serial.print("Discarded bytes: ");
    Serial.println(discardCnt);
  }
 
  // If we got to here then we have received a valid packet header.
  // Check to see if we got a valid packet type, if so store it. If not then
  // return false.
  receivedData[0] = readSerial1Byte();
  int idx;
  for (idx = 0; (idx < numPacketTypes) && (receivedData[0] != validPackets[idx]); idx++);
  if (idx == numPacketTypes) return false; // couldn't find valid packet type

// Read the transmitted data length
  receivedData[1] = readSerial1Byte();

// Compute how much remaining data to read
  // The data length transmitted includes 2 start of packet bytes, packet type byte,
  // and data length itself so subtract 4.
  byte remainingDataLength = receivedData[1] - 4;

for (byte i = 0; (i < remainingDataLength) && (i < maxBytes - 2); i++)
  {
    receivedData[i+2] = readSerial1Byte();
  }
  return true;
}

void displayPacket()
{
  // Determine packet size
  int idx;
  for (idx = 0; (idx < numPacketTypes) || (receivedData[0] != validPackets[idx]); idx++);
  if (idx == numPacketTypes)
  {
    // should never happen
    Serial.println("displayPacket(): Invalid packet type");
    return;
  }
 
  // Compute how much data to display
  // The data length transmitted includes 2 start of packet bytes so subtract 2.
  byte dataLength = receivedData[1] - 2;

// Display packet
  for (int i = 0; (i < dataLength) && (i < maxBytes); i++)
  {
    Serial.print(receivedData[i], HEX);
    Serial.print(" ");
  }
  Serial.println(" ");
}

8

Thanks for your sketch. However, it's not reading a packet.

For troubleshooting, I inserted 3 print statements. The 3rd print never gets reached. If I change:

if (startCnt == 2) break;
to:
if (startCnt == 1) break;

then the sketch will get past that point. I realize just one of the two "$" headers are now reconized. The sketch will notice the error further down the line.

It doesn't appear break is implemented even though startCnt gets to 2.
With: if (startCnt == 2) break; we get:

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.04.10 14:07:46 =~=~=~=~=~=~=~=~=~=~=~=
ChargerySniffer
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
No STARTBYTE reset counter
got STARTBYTE
got STARTBYTE

// These can be modified to accommodate changes in packets without having
// to change processing code.
const byte STARTBYTE = '

;
const byte validPackets[] = {0x56, 0x57};
const int numPacketTypes = sizeof(validPackets)/sizeof(validPackets[0]);

// Receive data buffer
const int maxBytes = 30;
byte receivedData[maxBytes];  // an array to store the received data

void setup()
{
  Serial.begin(115200);
  Serial.println("ChargerySniffer");
  Serial1.begin (115200);
}

void loop()
{
  // Get a packet and display it if it is valid
  if (getPacket())
  {
    displayPacket();
  }
  else
  {
    Serial.print("Invalid packet type received: ");
    Serial.println(receivedData[0], HEX);
  }
}

byte readSerial1Byte()
{
  while (!Serial1.available());
  return Serial1.read();
}

bool getPacket()
{
  int discardCnt = 0;

// Get start of packet
  while (true)
  {
    int startCnt = 0;

if (readSerial1Byte() == STARTBYTE)
    {
      // Increment start counter and see if this is the 2nd one
      Serial.println("got STARTBYTE");  //****************
      startCnt++;
      if (startCnt == 2) break; 
    }
    else
    {
      // Not a start byte so reset counter
      Serial.println("No STARTBYTE reset counter");  //************
      startCnt = 0;
      discardCnt++;
    }
  }

// If you are interested in how many bytes were discarded you can print or store
  // disacrdCnt variable here.
  // The first call of this function may have disacrded bytes. Subsequent calls
  // with discarded bytes may indicate software or serial line errors.
  Serial.println("discardCnt");  //*******************
  if (discardCnt > 0)
  {
    Serial.print("Discarded bytes: ");
    Serial.println(discardCnt);
  }

// If we got to here then we have received a valid packet header.
  // Check to see if we got a valid packet type, if so store it. If not then
  // return false.
  receivedData[0] = readSerial1Byte();
  int idx;
  for (idx = 0; (idx < numPacketTypes) && (receivedData[0] != validPackets[idx]); idx++);
  if (idx == numPacketTypes) return false; // couldn't find valid packet type

// Read the transmitted data length
  receivedData[1] = readSerial1Byte();

// Compute how much remaining data to read
  // The data length transmitted includes 2 start of packet bytes, packet type byte,
  // and data length itself so subtract 4.
  byte remainingDataLength = receivedData[1] - 4;

for (byte i = 0; (i < remainingDataLength) && (i < maxBytes - 2); i++)
  {
    receivedData[i+2] = readSerial1Byte();
  }
  return true;
}

void displayPacket()
{
  // Determine packet size
  int idx;
  for (idx = 0; (idx < numPacketTypes) || (receivedData[0] != validPackets[idx]); idx++);
  if (idx == numPacketTypes)
  {
    // should never happen
    Serial.println("displayPacket(): Invalid packet type");
    return;
  }

// Compute how much data to display
  // The data length transmitted includes 2 start of packet bytes so subtract 2.
  byte dataLength = receivedData[1] - 2;

// Display packet
  for (int i = 0; (i < dataLength) && (i < maxBytes); i++)
  {
    Serial.print(receivedData[i], HEX);
    Serial.print(" ");
  }
  Serial.println(" ");
}

9

Move the int startCnt = 0 outside of the loop like this:

  // Get start of packet
  int startCnt = 0;
  while (true)
  {
    if (readSerial1Byte() == STARTBYTE)
    {
      // Increment start counter and see if this is the 2nd one
      Serial.println("got STARTBYTE");  //****************
      startCnt++;
      if (startCnt == 2) break;   
    }
    else
    {
      // Not a start byte so reset counter
      Serial.println("No STARTBYTE reset counter");  //************
      startCnt = 0;
      discardCnt++;
    }
  }

Sorry about that...

10

I had a hunch "int startCnt = 0" needed to be moved up one notch.

It works great now! Life is (relatively) good again! Greatly appreciated. Now I can work on parcing and logging the data.
Gerry

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2020.04.11 08:22:42 =~=~=~=~=~=~=~=~=~=~=~=
ChargerySniffer
57 F D 98 2 0 0 0 98 0 98 5F E4  
56 16 C F9 C FC C FE C FA 0 0 0 0 0 58 0 21 5F A9  
57 F D 98 2 0 0 0 97 0 98 5F E3  
Discarded bytes: 6
57 F D 98 2 0 0 0 97 0 98 5F E3  
57 F D 98 2 0 0 0 97 0 98 5F E3  
56 16 C F9 C FC C FB C FA 0 0 0 0 0 5A 0 21 5F A8  
57 F D 98 2 0 0 0 97 0 98 5F E3  
57 F D 98 2 0 0 0 97 0 98 5F E3  
Discarded bytes: 6
56 16 C F7 C FC C FB C FC 0 0 0 0 0 5A 0 1F 5F A6  
57 F D 98 2 0 0 0 97 0 98 5F E3  
57 F D 98 2 0 0 0 96 0 98 5F E2  
56 16 C F9 C F9 C FB C FA 0 0 0 0 0 5A 0 1F 5F A3  
57 F D 98 2 0 0 0 96 0 98 5F E2  
57 F D 98 2 0 0 0 98 0 97 5F E3  
Discarded bytes: 6
56 16 C F6 C FC C FD C FA 0 0 0 0 0 58 0 22 5F A6  
57 F D 98 2 0 0 0 98 0 97 5F E3  
57 F D 98 2 0 0 0 98 0 98 5F E4  
57 F D 98 2 0 0 0 97 0 98 5F E3  
56 16 C F7 C FC C FB C FA 0 0 0 0 0 58 0 22 5F A5  
57 F D 98 2 0 0 0 97 0 98 5F E3  
Discarded bytes: 6
57 F D 98 2 0 0 0 97 0 97 5F E2  
56 16 C F9 C FC C FD C FC 0 0 0 0 0 5A 0 1F 5F AA  
57 F D 98 2 0 0 0 98 0 98 5F E4  
57 F D 98 2 0 0 0 98 0 98 5F E4  
56 16 C F9 C FC C FD C FA 0 0 0 0 0 58 0 1F 5F A6  
57 F D 98 2 0 0 0 98 0 97 5F E3  
Discarded bytes: 6
57 F D 98 2 0 0 0 98 0 97 5F E3  
56 16 C F9 C FB C FE C FA 0 0 0 0 0 58 0 21 5F A8  
57 F D 98 2 0 0 0 97 0 98 5F E3  
57 F D 98 2 0 0 0 97 0 98 5F E3  
57 F D 98 2 0 0 0 97 0 98 5F E3
11

Awesome! It’s hard to write code correctly without being able to debug it.

I guess the discarded bytes are those unidentified packets you showed in your original post.

12

ToddL1962:
Awesome! It’s hard to write code correctly without being able to debug it.

I'm really amazed. I've been struggling with that sketch for a while.

ToddL1962:
I guess the discarded bytes are those unidentified packets you showed in your original post.

The discarded bytes could be some of those unidentified packets shown in my first post. Something else could be going on. I'm not a digital or software guy, but I think the BMS signal is somewhat unstable. I've connected a Saleae Logic analyzer to the signal. The analyzer indicates quite a few "framing errors". I'm not sure what that means. Doesn't sound good?

I logged some data for the first time this morning. Out of about 300 measurements, at 10 second increments there were about 15 0x56 packets where the data is not correct. I'm going to use check sum to test for valid data. Here's a plot of this data. The 15 bad packets were first removed from the spreadsheed before plotting. Vbat and current are on the right hand secondary axis. Charging current is about 19A. Around the 30 min mark I had disconnected the charging source for a short time. At about 38 min, the BMS disconnected charging because max cell voltage was achieved. About 10 min later charging started up again.

Not sure why Cell4 voltage drops sometimes. I don't believe this is a packet error, but a BMS measurement error. Need to further investigate.

BMS1stData.jpg

13

Gerry48:
I'm really amazed. I've been struggling with that sketch for a while.

The discarded bytes could be some of those unidentified packets shown in my first post. Something else could be going on. I'm not a digital or software guy, but I think the BMS signal is somewhat unstable. I've connected a Saleae Logic analyzer to the signal. The analyzer indicates quite a few "framing errors". I'm not sure what that means. Doesn't sound good?

I logged some data for the first time this morning. Out of about 300 measurements, at 10 second increments there were about 15 0x56 packets where the data is not correct. I'm going to use check sum to test for valid data. Here's a plot of this data. The 15 bad packets were first removed from the spreadsheed before plotting. Vbat and current are on the right hand secondary axis. Charging current is about 19A. Around the 30 min mark I had disconnected the charging source for a short time. At about 38 min, the BMS disconnected charging because max cell voltage was achieved. About 10 min later charging started up again.

Not sure why Cell4 voltage drops sometimes. I don't believe this is a packet error, but a BMS measurement error. Need to further investigate.

If it were me, and you've already stated this, I would implement as many sanity checks as possible. 1) checksum, 2) data length. You know what the data length should be because it is always consistent. Compare that with what is being sent in the length byte.

If #1 and #2 don't clear up bad data you may have measurement issues in which case you'll have to come up with a scheme to detect and discard outlier readings. Could be a combination of thresholds, deadbands, sequential readings, etc.