Good morning.

This is my first attempt at any sort of software coding, so please try and look past any poor/strange uses of the code.  I'll get better with time - honest. Also note that my code is currently full of loads of mySerial.print and LCD.print commands. These are mostly for debug and testing. Most will be removed at some point.

My project is essentially an Arduino interface between the IBUS (data bus used to control the audio system and other non critical systems on older BMW cars) and an SPI controlled volume control circuit (based around the TI PGA2311 volume control IC). It listens for messages on the IBUS and adjusts the volume, responds to certain buttons and will eventually do a bit of audio source switching.

For the most part it actually works quite well, but falls over when the car transmits a long IBUS message. I don't actually need to respond to or act on any IBUS messages longer than 7 HEX bytes, but the car sends messages around the bus that can be over 20 HEX bytes. My code (below) currently works fine with 6 byte messages.
If I receive messages less than 6 bytes it ignores them because of the:

if (Serial.available()>=6)

If I receive messages longer than 6 bytes, it tries to read them before they have arrived and ends up missing them. This causes the LENGTH byte to read the wrong value and the code tries to read in serial that doesn't exist. My chkSUM routine then fails to calculate.

This is what I get if I receive a 7 byte message (44 05 BF 74 04 00 8E)

IBUSbyte = 44 5 BF 74 4 0 FF
Source= 44
Length= 5
IBUS checksum = 8E
IBUS checksum bad

It's failed to read the final byte, but my chkSUM routine has correctly calculated the checksum. It has also correctly identified the message as corrupt because the calculated chkSUM doesn't match the received chkSUM.

If I receive the same message again, I get the following:

IBUSbyte = 8E 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AF AF F8 FF F8 FF 1 0 1 0 0 0 0 0 2 0 0 2 3 B6 2 0 0 27 8 4 2 4 14 4 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Source= 8E
Length= 44
IBUS checksum = 0
IBUS checksum matches

No Match
IBUSbyte = 5 BF 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AF AF F8 FF F8 FF 1 0 1 0 0 0 0 0 2 0 0 2 3 B6 2 0 0 27 8 4 2 4 14 4 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 27 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 A6 2 0 0 E8 3 0 0 0 0 0 0 0 4E 1 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Source= 5
Length= BF
IBUS checksum = 0
IBUS checksum matches

No Match

From this, you can see that the 7th byte (8E)  it failed to read the first time, has come in as the source byte and the length byte (44) is what should have been the source byte. It goes down hill rapidly from there and the Arduino has to be reset to recover from this.

My main problem is that the IBUS messages don't have any start or end characters for me to detect. I need to read the first two bytes before I know the length of the message. I can then read in the rest of the message and process it.

For reference, this is the anatomy of an IBUS message.

IBUS protocol:

 Example:

 When key is inserted into the ignition lock these messages are sent.

 44 05 BF 74 04 00 8F // in - key detected
 44 05 BF 74 00 FF 75 // out - key removed

 Structure of an IBUS packet (Ref first IBUS packet above):

 1st byte is the Source ID - 0x44 - in this case the car immobilser module
 2nd byte is length of the packet excluding the first two bytes (source & length) - 0x05
 3rd byte is the destination ID - 0xBF - in this case the light control module
 4th byte onwards is the actual data - 0x74, 0x00, 0xFF
 Last byte is the checksum - 0x8F

 Checksum is generated by XOR of the entire packet excluding the checksum itself.

 HEX - Binary
 44  = 01000100
 05  = 00000101
 BF  = 10111111
 74  = 01110100
 04  = 00000100
 00  = 00000000
 XOR = 10001111 = 8F

This is the code that currently works fine with 6 byte messages:

void serialEvent() {
   
    if (Serial.available()>=6) { // there are bytes in the serial buffer to read
            IBUSbyte[0] = Serial.read(); // read in source byte.
            IBUSbyte[1] = Serial.read(); // read in length byte.
            LENGTH = IBUSbyte[1];
            if (IBUSbyte[0] == 0x50 || IBUSbyte[0] == 0x18 || IBUSbyte[0] == 0x68 || IBUSbyte[0] == 0xD7 || IBUSbyte[0] == 0x44 || IBUSbyte[0] == 0xC8 || IBUSbyte[0] == 0x80 && LENGTH <=7){   // ignore messages from some modules        
        for (int i = 2; i <= LENGTH+2; i++) {// read in rest of message.
            IBUSbyte[i] = Serial.read();
          }
            chkSUM(); // go off and see if checksum matches
          }
           
             else {
          debug();
          memset(IBUSbyte,0,sizeof(IBUSbyte));
          mySerial.println("No Match ");
         
     }
  }  
           
} // End of IBUS read routine.

I've spent more hours than I care to remember trying to improve on this, but my tiny brain just can't figure it out. If anyone can suggest any improvements or a better way to deal with this, I'd appreciate it.

I was going to post the full code, but it made the message longer than allowed.

Here's my chkSUM code:


void chkSUM() {
            checksumBYTE = 0;
            byteSTATE = 0;
        for (int i = 0; i < LENGTH+1; i++){
            checksumBYTE ^= IBUSbyte[i];}
        if (IBUSbyte[LENGTH + 1] == checksumBYTE){
            byteSTATE = 1;}
        else if (IBUSbyte[LENGTH + 1] != checksumBYTE){
            byteSTATE = 0;}
             
        if  (byteSTATE == 1) {
             compare();
           }
        else if (byteSTATE == 0) {
            debug();
            checksumBYTE = 0;
            memset(IBUSbyte,0,sizeof(IBUSbyte));
           
     }
}

Many thanks,

Ian.

Hi Robin,

Do you know the longest possible message?

I don't know the theoretical maximum length, but the longest I've ever seen is 37 hex bytes long:

D0 23 3F A0 00 40 FE FE 00 00 00 00 08 AA 00 00 00 00 00 54 00 00 F9 00 00 00 00 00 00 00 00 00 00 00 00 00 23

Why not just receive all messages (regardless of length) and just ignore some of them?

I guess that's ultimately what I'm trying to do - just not very well. 
I've figured out SPI, Rotary encoders, and all sorts of other good stuff, but simply reading in a string of serial data appears to be beyond me. Unfortunately, the whole project is dead in the water until I can figure it out.

How often are messages transmitted?

It varies, but there's quite a lot of traffic. Here's a small section from an IBUS log file with time stamps.

2014-04-14 17:07:55.015:  D0 07 BF 5C FF FF FF 00 CB
2014-04-14 17:07:55.015:  ! LCM  --> GLO : Light dimmer, Data="FF FF FF 00"
2014-04-14 17:07:58.500:  68 03 18 01 72
2014-04-14 17:07:58.500:  ! RAD  --> CDC : Device status request
2014-04-14 17:07:58.515:  18 04 FF 02 00 E1
2014-04-14 17:07:58.515:  ! CDC  --> LOC : Device status ready,
2014-04-14 17:07:58.531:  18 0A 68 39 00 02 00 3F 00 07 01 78
2014-04-14 17:07:58.531:  ! CDC  --> RAD : CD_status, Stop  Request=Pause  CD=7  Track=1  CDs_Loaded=P1,P2,P3,P4,P5,P6
2014-04-14 17:08:04.468:  E8 05 D0 59 50 00 34
2014-04-14 17:08:04.468:  ! RLS  --> LCM : Light control status, Control: Turn_lights_off Sensor_OK Level_Signal_invalid Reason:Data="50 00"
2014-04-14 17:08:08.031:  68 03 18 01 72
2014-04-14 17:08:08.031:  ! RAD  --> CDC : Device status request
2014-04-14 17:08:08.046:  18 04 FF 02 00 E1
2014-04-14 17:08:08.046:  ! CDC  --> LOC : Device status ready,
2014-04-14 17:08:08.046:  18 0A 68 39 00 02 00 3F 00 07 01 78
2014-04-14 17:08:08.046:  ! CDC  --> RAD : CD_status, Stop  Request=Pause  CD=7  Track=1  CDs_Loaded=P1,P2,P3,P4,P5,P6
2014-04-14 17:08:10.000:  D0 07 BF 5C FF FF FF 00 CB
2014-04-14 17:08:10.000:  ! LCM  --> GLO : Light dimmer, Data="FF FF FF 00"
2014-04-14 17:08:14.484:  E8 05 D0 59 50 00 34
2014-04-14 17:08:14.484:  ! RLS  --> LCM : Light control status, Control: Turn_lights_off Sensor_OK Level_Signal_invalid Reason:Data="50 00"
2014-04-14 17:08:17.546:  68 03 18 01 72
2014-04-14 17:08:17.546:  ! RAD  --> CDC : Device status request
2014-04-14 17:08:17.562:  18 04 FF 02 00 E1
2014-04-14 17:08:17.562:  ! CDC  --> LOC : Device status ready,
2014-04-14 17:08:17.593:  18 0A 68 39 00 02 00 3F 00 07 01 78
2014-04-14 17:08:17.593:  ! CDC  --> RAD : CD_status, Stop  Request=Pause  CD=7  Track=1  CDs_Loaded=P1,P2,P3,P4,P5,P6
2014-04-14 17:08:18.531:  68 05 18 38 03 00 4E
2014-04-14 17:08:18.531:  ! RAD  --> CDC : CD_control, Play

It's a good idea to write a short sketch that just focuses on the problem at hand. Then you can post the whole sketch here and get useful advice without lazy me having to wade through lots of stuff that you know how to do.

Even if you won't/can't write a short sketch post a complete sketch here (not just snippets) so we can see all of it.

Try this for size. It's the smallest I could make it while still remaining functional.

Here's some sample IBUS messages that it's happy with.
50 04 68 32 11 1F
50 04 68 32 10 1E

These are ignored, but the software handles it fine.
50 04 68 3B 02 05
50 04 68 3B 22 25

These will upset things.
44 05 BF 74 04 00 8E
44 05 BF 74 00 FF 75
C8 0F 80 23 42 32 42 6C 75 65 20 52 6F 6F 6D 20 15
D0 23 3F A0 00 40 FE FE 00 00 00 00 08 AA 00 00 00 00 00 54 00 00 F9 00 00 00 00 00 00 00 00 00 00 00 00 00 23

#include <Wire.h>
  #include <SoftwareSerial.h>

SoftwareSerial mySerial(7, 8); // 7 is Rx, 8 is Tx
byte KEY_IN [7] = { 0x44 , 0x05 , 0xBF , 0x74 , 0x04 , 0x00 , 0x8E }; // Ignition key in
byte KEY_OUT [7] = { 0x44 , 0x05 , 0xBF , 0x74 , 0x00 , 0xFF , 0x75 }; // Ignition key out
byte VOL_UP [6] = { 0x50 , 0x04 , 0x68 , 0x32, 0x11 , 0x1F }; // Steering wheel Volume Up
byte VOL_DOWN [6] = { 0x50 , 0x04 , 0x68 , 0x32, 0x10 , 0x1E }; // Steering wheel Volume Down
byte LENGTH;
byte checksumBYTE;
byte IBUSbyte[20];
boolean byteSTATE = false;
int IBUSevent = 0; //boolean value set 1 if IBUS message detected
int EventID = 0; // Event number (for case select)
int CS_LWAKE = 4;

void setup()
{
  Serial.begin(9600, SERIAL_8E1);
  mySerial.begin(115200);
  mySerial.println("--Digital Dock Volume Controller--");
  pinMode (CS_LWAKE, OUTPUT); // initialize pin.
  digitalWrite (CS_LWAKE, HIGH); // write pin high.
}

void loop(){
 
   if (IBUSevent == 1){
    mySerial.println();
     
    switch (EventID){
      case 100:
        mySerial.println("UNKNOWN KEY");
        break;
       
       case 1:
        mySerial.println("Key IN");
        break;
       
      case 2:
        mySerial.println("Key OUT");
        break;
       
      case 3:
         mySerial.println("VOLup");
        break;   
         
      case 4:
        mySerial.println("VOLdown");
        break;         
    }
        IBUSevent = 0;
   
  }
       

} // END OF MAIN LOOP

void serialEvent() {
   
     if (Serial.available()>=6) { // there are 6 or more bytes in the serial buffer to read
             IBUSbyte[0] = Serial.read(); // read in source byte and store in IBUSbyte location 0
             IBUSbyte[1] = Serial.read(); // read in length byte and store in IBUSbyte location 1
             LENGTH = IBUSbyte[1];
             if (IBUSbyte[0] == 0x50 || IBUSbyte[0] == 0x44 && LENGTH <=7){   // ignore messages from some modules         
         for (int i = 2; i <= LENGTH+2; i++) {// read in rest of message.
             IBUSbyte[i] = Serial.read();
             }
               chkSUM(); // go off and see if checksum matches
             }
             else
             {
               debug();
               mySerial.println("No Match ");
             }
   } 
             
} // End of IBUS read routine.

void compare(){
     EventID = 100;
     if (byteSTATE == 1) {
     if(memcmp(IBUSbyte, KEY_IN, 7) == 0 ) {EventID = 1;}
     if(memcmp(IBUSbyte, KEY_OUT, 7) == 0 ){EventID = 2;}
     if(memcmp(IBUSbyte, VOL_UP, 6) == 0 ){EventID = 3;}
     if(memcmp(IBUSbyte, VOL_DOWN, 6) == 0 ){EventID = 4;}
     IBUSevent=1;
     }
      else
      {
        IBUSevent=0;
      }
     
}
     
void chkSUM() {
             checksumBYTE = 0;
             byteSTATE = 0;
         for (int i = 0; i < LENGTH+1; i++){ // read in stored bytes and calculate XOR
             checksumBYTE ^= IBUSbyte[i];}
         if (IBUSbyte[LENGTH + 1] == checksumBYTE){ // see if calculated XOR matches received XOR
             byteSTATE = 1;}
         else if (IBUSbyte[LENGTH + 1] != checksumBYTE){
             byteSTATE = 0;}
             
         if  (byteSTATE == 1) // if it matches, go and compare the received byte with the ones stored above
            {
              compare();
            }
         else if (byteSTATE == 0) // if it doesn't match, run debug() and display some diagnostic info
            {
             debug();
             checksumBYTE = 0;
             }
}


void debug(){

         mySerial.print("IBUSbyte = ");
         for (int i = 0; i <= LENGTH+1; i++) {         
              mySerial.print(IBUSbyte[i], HEX);
              mySerial.print(" ");
              }
              mySerial.println();
              mySerial.print("Source= ");
              mySerial.println(IBUSbyte[0], HEX);
              mySerial.print("Length= ");
              mySerial.println(IBUSbyte[1], HEX);
              mySerial.print("IBUS checksum = ");
              mySerial.println(checksumBYTE, HEX);
          if (IBUSbyte[LENGTH + 1] == checksumBYTE) {
              mySerial.println("IBUS checksum matches "); }
          else if (IBUSbyte[LENGTH + 1] != checksumBYTE){
              mySerial.println("IBUS checksum bad "); }
              mySerial.println();
              checksumBYTE = 0;
   
}

It seems like you are reading the IBUS with hardware serial and looking at the results with software serial. Is that correct? Where is the software serial output appearing?

Hi Robin,

I'm reading the IBUS with hardware serial because I need SERIAL_8E1 for the IBUS. I don't appear to be able to do that with software serial. IBUS communicates at 9600 baud.

Serial.begin(9600, SERIAL_8E1);

I'm only using software serial to view the results (in PuTTY) for debugging etc. This can be at any baud rate, 115200 is not a requirement. I won't need to use software serial at all once it's all working. Everything will be done by listening to the IBUS and acting on specific messages.

I only discovered serialEvent a few days ago and I suggest you don't use it as who knows when it decides to trigger itself.

So did I. It seemed like a nice way to call the IBUS read routine. I was doing it a different way before, but it's not caused me any problems so far.

If this were my project I think I would organize the program like this

Code: [Select]

void loop() {
  readIbus();  // a function that just reads in data and saves it in a buffer
  interpretData(); // a function that assesses what's in the buffer and sets a variable if something
                                needs to be done
  doSomethingWithTheData();  // a function that does stuff when necessary
}

You can easily test if there is something in the serial buffer with serial.available() without needing serialEvent to start the ball rolling.

It's the first bit I'm struggling with. I've tried this sort of thing, but I just don't seem to get any data in my buffer.

int i = 0;
     while(Serial.available()>0){
       byte INbyte = Serial.read();
       IBUSbyte[i++] = INbyte;
      }

I'm probably being really thick, but I can't figure it out. Once it's in my buffer, I'm sure I can work with it.

Do the IBUS messages have formal start and end characters? If so it will make reading much easier.

Rather than say more, and confuse matters, I will wait for your response.

I'm afraid not. They just come in one after the other with no start or end characters.

Cheers,

Ian.

That serial code looks like it should work. How do you know it's not working? You haven't shown where the variables are declared so I can't say if you have an error there.

I would write a short sketch that just reads that and prints the buffer contents from time to time.

This would then become the function I called readIbus().

Hi Robin,

I just took your advice and wrote a new sketch using the serial code that I posted above. I'm not sure what was going on in my original sketch when using this code, but in the new sketch, it works a treat. I think this is why I was so confused. It should have worked, but didn't.

If I was using the code structure I suggested I would use IF (serial.available > 0) rather than WHILE so that the code doesn't sit reading characters at the slow baud rate (and sorry for getting my baud rates mixed up - a nice slow 9600 is best).

I'll try that. Thanks.

Have you an FTDI cable for software serial?

Similar. It's something like this:

http://forum.arduino.cc/index.php?topic=66416.0

Thanks for your help 

Hi Robin,

this is still fighting with me. Should this work ?

#include <Wire.h>
#include <SoftwareSerial.h>

SoftwareSerial mySerial(7, 8); // 7 is Rx, 8 is Tx
byte IBUSbyte[40];
boolean IBUSprocess = false;



void setup()
{
  Serial.begin(9600, SERIAL_8E1);
  mySerial.begin(115200);
  mySerial.println("--Digital Dock Volume Controller--");
 
 
}

void loop() {
 
  readIBUS();
  interpretData();
  processData();
 
}     

void readIBUS() {
       int i = 0;
     if (Serial.available()>0){
       byte INbyte = Serial.read();
       IBUSbyte[i++] = INbyte;
      }
}


void interpretData() {
 
     if (IBUSbyte[0] == 0x50 || IBUSbyte[0] == 0x44) {
       IBUSprocess = true;}
       
     else
        {
         IBUSprocess = false;
        }
}

void processData() {
 
if (IBUSprocess == true) {
       
              mySerial.print(IBUSbyte[0], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[1], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[2], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[3], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[4], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[5], HEX);
              mySerial.println(" ");
              }
 
      else
         {
         
         }   
}

If I send - 44 05 BF 74 00 FF 75 - all I get printed back is 44 00 00 00 00 00
if I send - 50 04 68 32 11 1F -  - all I get printed back is 50 00 00 00 00 00

It only seems to be reading the first byte of the message and ignoring the rest. I'm not sure what's different between this and what I tried earlier, but this is the sort of issue I was having when I tried it in my original code.

Thanks again,

Ian.

A few thoughts come to mind ...

Is there any risk of receiving more than 40 chars - if so strange things will happen when ibusProcess and subsequent memory locations are over written.

I can't say for sure, but I don't think so. I can increase the array size if necessary.

I suspect there are two problems. First you have "int i = 0" in readIBUS() which means every character is saved in element 0. This value needs to be set to 0 only when you are ready to receive a new message. Second, it is printing the contents after a single character is received. It is probably receiving properly.

Good point. I was forgetting that  readIBUS() only ran once. At the time (it was late) I was thinking the 'if' loop inside readIBUS() would run until no more serial was available, and then return to the main loop. Now corrected and working better.

You need to organize things so that "IBUSprocess" is only true when a complete message is received - perhaps when a Cr or Lf is detected. There must be something that distinguishes one message from the next. If you don't know what it is I would just record 500 bytes and print them all out in Hex to look for a pattern.

I think this is where I'm struggling. There really is no Cr or Lf. If I repeatedly send the same message ( 44 05 BF 74 00 FF 75 ), I just get a repeating string with no gaps, spaces or other characters.

445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75

I believe the key is the length bit (second bit). Given this, you can calculate the length of the incoming message, but you need to receive this before you can act on it. This is what I was attempting to do in my original sketch, but couldn't get it flexible enough to cope with variable length messages.

Cheers,

Ian.

how do you know that you're seeing the first byte of a message when you start? i.e. what if there is a message being transmitted on the bus when you power up the arduino? Are there periods of quiet on the bus? Are the source ids unique bytes that aren't used anywhere else in the protocol - I doubt it.

All valid points. I think the hardware sorts out the power up side of things. I'm using a Microchip MCP2025 LIN BUS transceiver chip to convert the IBUS signals to TTL levels ( http://www.microchip.com/wwwproducts/Devices.aspx?dDocName=en558481 ). There are periods of quiet, but nothing of a fixed length as far as I can see. Source ID bytes are also used in the rest of the message, so that can't be relied upon either.

I think the answer is going to be a combination of the length byte and the checksum. Given both of these, you can tell if the message is a valid one.

Cheers,

Ian.

I believe the key is the length bit (second bit).

When I asked about marker bytes earlier you said there were none. It's a bit hard to give useful advice without the full picture.

What is the byte before the length byte? It doesn't seem to be constant. How do you know it is the first byte of a sequence and the byte following it will be a length byte.

How do you know the byte before the length byte isn't actually the last byte (perhaps a checksum) for the previous message?

...R