Go Down

Topic: Read specific records from data stream, every 10 seconds... (Read 183 times) previous topic - next topic

MaxG

I have tried for hours to figure out how I can speed limit reading specific messages from a CAN bus data stream.  :smiley-sad-blue:

I can read the messages I am after; they come at a rate for 4 Hz and Serial.print() these in the monitor.
However, I need to pass the messages on to another system and want to limit the flow of information.
I can limit querying the bus to any time I want (here 10 seconds), but, I always end up getting the same message Id (301), and not the other five Ids I am after.

I thought about state machines, then a "do until" would be nice, but this does not translate to C. E.g. loop until id = 301.


So the question is: How can read the bus to get all IDs {301,302,303,304,30,40}, then wait some time, say 10 sec, then do it again.
Actually more precise, the IDs come in succession, but I want to read ID 301, then convert, send it off, then move to the next ID (302)... 303, etc. then wait 10 sec, and do it again.
Hope this makes sense.

Code: [Select]

void loop() {

  every ten seconds, do {
    read ID 301
    convert
    send
    read 302
    ...
  }
}



The problem seems to be the 10 second timer; as it is written it read one ID, waits 10 sec and then reads 301 again, but not the others.


Here the code dummied up that reads 301 only.:

Code: [Select]

#include <Arduino.h>
#include <mcp_can.h>
#include <SPI.h>

const unsigned long BAUD_RATE_HW            = 115200;                 // baud rate for hardware serial port
const unsigned long CAN_READ_INTERVAL       =  10000;                 // [ms]
const byte PIN_CAN_SPI_CS                   =      9;                 // CS (Chip Select) pin
unsigned long receiveTime                   =      0;
unsigned char dataLength                    =      0;
unsigned char receiveBuffer[8];
long unsigned int packetId;

MCP_CAN CAN(PIN_CAN_SPI_CS);                      // Construct a MCP_CAN Object and set Chip Select to 10.


// Prototypes
void readCAN();
void operatingStatus();
void batteryCurrent();
void getCellVoltages();

void setup() {
  Serial.begin(BAUD_RATE_HW);                   // start hardware serial port with 9600 bps
  while (CAN_OK != CAN.begin(CAN_250KBPS,MCP_8MHz)) {     // init can bus : baudrate = 250K
    Serial.println(F("CAN BUS Module: failed to initialise; retrying..."));
    delay(500);
  }
}

void loop() {
  static unsigned long lastCanRead = 0;
  unsigned long timeStamp = millis();             // memorise current time

  if (timeStamp - lastCanRead > CAN_READ_INTERVAL) {
    lastCanRead = timeStamp;
    Serial.println(F("readCAN"));
    readCAN();
  }
}

void readCAN() {

  if (CAN_MSGAVAIL == CAN.checkReceive()) {       // check if data coming
    receiveTime = millis();

    CAN.readMsgBuf(&dataLength, receiveBuffer);   // read data,  len: data length, buf: data buf
    packetId = CAN.getCanId();

    switch (packetId) {
      case 301:                                   // cell voltages  1 to  4
      case 302:                                   // cell voltages  5 to  8
      case 311:                                   // cell voltages  9 to 12
      case 312:                                   // cell voltages 13 to 16
        getCellVoltages();
        break;

      case 30:                                    // Broadcast Status
        operatingStatus();
        break;

      case 40:                                    // Battery Current
        batteryCurrent();
        break;

      default:                                    // anything else
        break;
    }
  }
}

void getCellVoltages() {
  Serial.println("getCellVoltages");
}

void operatingStatus() {
  Serial.println("operatingStatus");
}

void batteryCurrent() {
  Serial.println("batteryCurrent");
}


Mainly using UNOs. Everything needs to be defined.

wildbill

How do you know that you're only receiving 301? In your example, you are only printing "getCellVoltages" whichever of the four Ids you see. I'd print the ID somewhere.

Robin2

I can read the messages I am after; they come at a rate for 4 Hz and Serial.print() these in the monitor.
However, I need to pass the messages on to another system and want to limit the flow of information.
Why not read them all and pick out the items of interest to store in appropriate variables whenever they arrive. Then, at whatever interval you choose, sent the content of those variables on to somewhere else. In other words, separate the collection from the sending. And you will always be sending the most recent values that you received.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

MaxG

How do you know that you're only receiving 301? In your example, you are only printing "getCellVoltages" whichever of the four Ids you see. I'd print the ID somewhere.
Well, I tested the code iteratively.
Send dummy > receive dummy = all boards / components are working.
Read real CAN bus... works
Read and convert CAN bus data ... works

results:
301 3.21 3.22 3.21 3.20
302 3.22 3.21 3.20 3.21
303 3.21 3.20 3.21 3.22
304 3.20 3.21 3.21 3.20
30 99
40 1

These values pop in 4 times per second.
The target system only needs the data once per 10 seconds.
The target system cannot consume the data at 4 Hz.

The data stream pulses: 5 ids in milliseconds as a block, 4 blocks per second.


I am still thinking and ponder, how can I get
loop {
 301 convert send wait 1 sec
 302 convert send wait 1 sec
 303 convert send wait 1 sec
 30 convert send wait 1 sec
 40 convert send wait 1 sec
 then sleep for ten sec
}

I drew this up and it get convoluted, as I cannot seem to grasp the pattern.

I basically need to loop reading the CAN bus
plug 301
plug 302
...
then stop processing for ten secs and start plugging :)

I can post the code 600 lines, but it does not help (as in adding any value for solving the problem at hand.
The issue is to read one msg after the other at a lower pace than the data that's coming in.
Which IMHO lives in the "loop" routine.




Mainly using UNOs. Everything needs to be defined.

MaxG

Why not read them all and pick out the items of interest to store in appropriate variables whenever they arrive.
...
I am tight in memory...

I understand what you're suggesting... but do not have the memory for it.
I'd like to send one of each msg with a 1 sec spacing, then sleep for 10 sec.
The order of msgs does not matter, as long as I get all 6 msg, then sleep for 10.

The current program (processing the data properly at 4 Hz and Serial.printing it compiles with:
Code: [Select]

DATA:    [======    ]  60.3% (used 1234 bytes from 2048 bytes)
PROGRAM: [=======   ]  74.3% (used 23958 bytes from 32256 bytes)


It's an UNO reading the CAN bus, requires SPI, runs Ethernet, and MQTT on top of it.
the 301 to 304 msgs have some 100 bytes each.
Mainly using UNOs. Everything needs to be defined.

wildbill

If you can't find a bit more RAM to follow Robin2's suggestion, I'd use an array of booleans and an enum to tell me which packet ID was associated with each.

Set them all false and start reading packets. When you see one that hasn't been sent yet (according to the array), set the array element true, process the packet, wait until it's a second since you sent the last one and then send it.

This assumes that you don't care which order the packets are sent in - it'll just pick any one that it "owes" the target system.

When all are sent, reset the booleans and wait ten seconds.

If memory is really really tight, you can pack those booleans into a couple of bytes and use bit twiddling operations to get at them.

Robin2

I am tight in memory...

I understand what you're suggesting... but do not have the memory for it.
I don't see how you would need any extra memory for my suggestion. Presumably you will already have the variables to hold the items of interest?

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

MaxG

... use an array of booleans and an enum to tell me which packet ID was associated with each.

Set them all false and start reading packets...

This assumes that you don't care which order the packets are sent in...
...
When all are sent, reset the booleans and wait ten seconds.
Thanks, this sounds like a plan!
I have started using an array since posting, but could not figure what to do with it. Too many "if"s and checks.
enum booleans for each msg ID, great... I can see it.

Order of msgs does not matter, as long as I get an update on the group of IDs at around the same (say) 5 seconds window.

I'll post what I come up with.
Mainly using UNOs. Everything needs to be defined.

MaxG

I thought it helps to post the expected output...
We can see the msgs at 4Hz.

Code: [Select]

08/24/19 13:59:08.029 ArgyleCourt/Shed/Tower/Zeva16/Notification Arduino_CAN_BMS|INFO|has rebooted
08/24/19 13:59:08.034 ArgyleCourt/Shed/Tower/Zeva16/MAC 55:4E:4F:30:30:31

08/24/19 13:59:08.043 ArgyleCourt/Shed/Tower/Zeva16/Status 30,2,0,557,4000,40
08/24/19 13:59:08.053 ArgyleCourt/Shed/Tower/Zeva16/Voltages 301,3495,3491,3496,3495
08/24/19 13:59:08.088 ArgyleCourt/Shed/Tower/Zeva16/Voltages 301,3493,3494,3495,3496
08/24/19 13:59:08.094 ArgyleCourt/Shed/Tower/Zeva16/Voltages 302,3491,3305,3588,3506
08/24/19 13:59:08.155 ArgyleCourt/Shed/Tower/Zeva16/Voltages 311,3495,3501,3335,3555
08/24/19 13:59:08.163 ArgyleCourt/Shed/Tower/Zeva16/Voltages 312,3495,3495,3491,3496
08/24/19 13:59:08.216 ArgyleCourt/Shed/Tower/Zeva16/Current 40,0

08/24/19 13:59:08.284 ArgyleCourt/Shed/Tower/Zeva16/Status 30,2,0,557,4000,40
08/24/19 13:59:08.348 ArgyleCourt/Shed/Tower/Zeva16/Voltages 301,3494,3491,3490,3495
08/24/19 13:59:08.358 ArgyleCourt/Shed/Tower/Zeva16/Voltages 302,3491,3304,3591,3505
08/24/19 13:59:08.410 ArgyleCourt/Shed/Tower/Zeva16/Voltages 311,3492,3498,3333,3553
08/24/19 13:59:08.420 ArgyleCourt/Shed/Tower/Zeva16/Voltages 312,3490,3496,3495,3493
08/24/19 13:59:08.466 ArgyleCourt/Shed/Tower/Zeva16/Current 40,0

08/24/19 13:59:08.531 ArgyleCourt/Shed/Tower/Zeva16/Status 30,2,0,557,4000,40
08/24/19 13:59:08.593 ArgyleCourt/Shed/Tower/Zeva16/Voltages 301,3491,3496,3491,3492
08/24/19 13:59:08.599 ArgyleCourt/Shed/Tower/Zeva16/Voltages 302,3494,3303,3589,3502
08/24/19 13:59:08.656 ArgyleCourt/Shed/Tower/Zeva16/Voltages 311,3492,3503,3332,3553
08/24/19 13:59:08.663 ArgyleCourt/Shed/Tower/Zeva16/Voltages 312,3493,3491,3491,3493
08/24/19 13:59:08.717 ArgyleCourt/Shed/Tower/Zeva16/Current 40,0

08/24/19 13:59:08.780 ArgyleCourt/Shed/Tower/Zeva16/Status 30,2,0,557,4000,40
08/24/19 13:59:08.838 ArgyleCourt/Shed/Tower/Zeva16/Voltages 301,3495,3493,3495,3492
08/24/19 13:59:08.844 ArgyleCourt/Shed/Tower/Zeva16/Voltages 302,3494,3304,3589,3506
08/24/19 13:59:08.905 ArgyleCourt/Shed/Tower/Zeva16/Voltages 311,3492,3503,3333,3553
08/24/19 13:59:08.912 ArgyleCourt/Shed/Tower/Zeva16/Voltages 312,3494,3494,3493,3493
08/24/19 13:59:08.966 ArgyleCourt/Shed/Tower/Zeva16/Current 40,0


08/24/19 13:59:09.025 ArgyleCourt/Shed/Tower/Zeva16/Status 30,2,0,557,4000,40
08/24/19 13:59:09.094 ArgyleCourt/Shed/Tower/Zeva16/Voltages 301,3494,3494,3492,3494
08/24/19 13:59:09.100 ArgyleCourt/Shed/Tower/Zeva16/Voltages 302,3492,3304,3588,3509
08/24/19 13:59:09.159 ArgyleCourt/Shed/Tower/Zeva16/Voltages 311,3497,3499,3332,3551
08/24/19 13:59:09.169 ArgyleCourt/Shed/Tower/Zeva16/Voltages 312,3495,3492,3494,3497
08/24/19 13:59:09.220 ArgyleCourt/Shed/Tower/Zeva16/Current 40,0

... and so forth


I still have to figure out how to throttle the msgs, but also have to admit I had not much quiet time to do so yet.
Mainly using UNOs. Everything needs to be defined.

MaxG

Solved... it was easier than I thought, and wonder why I sat there for hours trying to figure this out.
It turned out that wildbill's booleans triggered a thought... with the solution being embarrassing simple.  :o

In essence, I added this piece of code in the loop():
Code: [Select]

    if (keepReadingCAN == true) {
      readCAN();
    }

    if (timeStamp - lastCanRead > CAN_READ_INTERVAL) {
      lastCanRead = timeStamp;
      keepReadingCAN = true;
    }


and since the msgs do come in order, all I had to do is adding keepReadingCAN = false in the readCAN():
Code: [Select]

void readCAN() {

  if (CAN_MSGAVAIL == CAN.checkReceive()) {       // check if data coming
    receiveTime = millis();

    CAN.readMsgBuf(&dataLength, receiveBuffer);   // read data,  len: data length, buf: data buf
    packetId = CAN.getCanId();

    switch (packetId) {

      case 30:                                    // Broadcast Status (Tx)
        operatingStatus();
        break;

      case 40:                                    // Battery Current (Rx)
        batteryCurrent();
        break;

      case 301:                                   // cell voltages  1 to  4
      case 302:                                   // cell voltages  5 to  8
      case 311:                                   // cell voltages  9 to 12
        getCellVoltages();
        break;

      case 312:                                   // cell voltages 13 to 16
        getCellVoltages();
        keepReadingCAN = false;                   // got my block of msgs, done reading
        break;

      default:                                    // anything else
        break;
    }
  }
}


Thank you for your support!
Mainly using UNOs. Everything needs to be defined.

wildbill

I'd suggest you test it very thoroughly: you're heavily reliant on getting the 312 packet. If you miss it for some reason - corruption, hiccup on the CAN bus, delay in your program - it will mean that you send all the other values again. Does that matter?

I notice in your test data that you got two 301 packets in a row too, do you care?

Personally I'd rather have tighter control of what gets sent, but without knowing what the target system is doing it's hard to say whether it's important.

MaxG

I'd suggest you test it very thoroughly: you're heavily reliant on getting the 312 packet. If you miss it for some reason - corruption, hiccup on the CAN bus, delay in your program - it will mean that you send all the other values again. Does that matter?

I notice in your test data that you got two 301 packets in a row too, do you care?

Personally I'd rather have tighter control of what gets sent, but without knowing what the target system is doing it's hard to say whether it's important.
I ran this code, w/o sending mqtt msgs, simply Serial.print-ing on the monitor, and the 'blocks' (of msgs) came in sequentially and once per ID.

The technology we're looking at is a 20kWh battery with 16 3.2V LiFePo4 cells, monitored by a battery monitoring system (BMS from Zeva), which sends this data to a CAN bus connected LCD monitor. IT updates the monitor values at 4Hz.

I plugged into the CAN bus, with an MCP2515 board connected to an UNO. It has an Ethernetshield, and uses MQTT to capture the CAN msgs, decodes them, and sends the data to the target system.

The target system is openHAB (a home automation bus), which could digest the data at that speed. However, the frequency of 'updates' is not required, and only creates noise as well as load on the system to no benefit.

However, I use openHAB to graph the individual battery cells, and 1 minute msg intervals are sufficient for this purpose.
The graphs will provide an early indicator if a cell is going to fail, allowing for pro-active remediation. E.g. replacing the cell if need be. The other benefit of openHAB is to use its full palette of features; e.g. sending email on specified events, triggering actions, etc.

The code with disabled DEBUG compiles with this memory usage:
Code: [Select]

DATA:    [=====     ]  49.4% (used 1011 bytes from 2048 bytes)
PROGRAM: [=======   ]  66.5% (used 21466 bytes from 32256 bytes)


... which is not bad for what the UNO is tasked with.  :)

Yes, testing is a chapter on its own... and I will monitor the output for a while.
And yes, I like your boolean array... the current solution was the easiest for me to implement right now.

Again, thank you for your input, which I do appreciate.
Mainly using UNOs. Everything needs to be defined.

Go Up