Struggling with my BMW IBUS projects. Serial.read related issues.

Good morning.

This is my first attempt at any sort of software coding, so please try and look past any poor/strange uses of the code. I'll get better with time - honest. Also note that my code is currently full of loads of mySerial.print and LCD.print commands. These are mostly for debug and testing. Most will be removed at some point.

My project is essentially an Arduino interface between the IBUS (data bus used to control the audio system and other non critical systems on older BMW cars) and an SPI controlled volume control circuit (based around the TI PGA2311 volume control IC). It listens for messages on the IBUS and adjusts the volume, responds to certain buttons and will eventually do a bit of audio source switching.

For the most part it actually works quite well, but falls over when the car transmits a long IBUS message. I don't actually need to respond to or act on any IBUS messages longer than 7 HEX bytes, but the car sends messages around the bus that can be over 20 HEX bytes. My code (below) currently works fine with 6 byte messages.
If I receive messages less than 6 bytes it ignores them because of the:

if (Serial.available()>=6)

If I receive messages longer than 6 bytes, it tries to read them before they have arrived and ends up missing them. This causes the LENGTH byte to read the wrong value and the code tries to read in serial that doesn't exist. My chkSUM routine then fails to calculate.

This is what I get if I receive a 7 byte message (44 05 BF 74 04 00 8E)

IBUSbyte = 44 5 BF 74 4 0 FF
Source= 44
Length= 5
IBUS checksum = 8E
IBUS checksum bad

It's failed to read the final byte, but my chkSUM routine has correctly calculated the checksum. It has also correctly identified the message as corrupt because the calculated chkSUM doesn't match the received chkSUM.

If I receive the same message again, I get the following:

IBUSbyte = 8E 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AF AF F8 FF F8 FF 1 0 1 0 0 0 0 0 2 0 0 2 3 B6 2 0 0 27 8 4 2 4 14 4 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Source= 8E
Length= 44
IBUS checksum = 0
IBUS checksum matches

No Match
IBUSbyte = 5 BF 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 AF AF F8 FF F8 FF 1 0 1 0 0 0 0 0 2 0 0 2 3 B6 2 0 0 27 8 4 2 4 14 4 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 27 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 A6 2 0 0 E8 3 0 0 0 0 0 0 0 4E 1 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Source= 5
Length= BF
IBUS checksum = 0
IBUS checksum matches

No Match

From this, you can see that the 7th byte (8E) it failed to read the first time, has come in as the source byte and the length byte (44) is what should have been the source byte. It goes down hill rapidly from there and the Arduino has to be reset to recover from this.

My main problem is that the IBUS messages don't have any start or end characters for me to detect. I need to read the first two bytes before I know the length of the message. I can then read in the rest of the message and process it.

For reference, this is the anatomy of an IBUS message.

IBUS protocol:

Example:

When key is inserted into the ignition lock these messages are sent.

44 05 BF 74 04 00 8F // in - key detected
44 05 BF 74 00 FF 75 // out - key removed

Structure of an IBUS packet (Ref first IBUS packet above):

1st byte is the Source ID - 0x44 - in this case the car immobilser module
2nd byte is length of the packet excluding the first two bytes (source & length) - 0x05
3rd byte is the destination ID - 0xBF - in this case the light control module
4th byte onwards is the actual data - 0x74, 0x00, 0xFF
Last byte is the checksum - 0x8F

Checksum is generated by XOR of the entire packet excluding the checksum itself.

HEX - Binary
44 = 01000100
05 = 00000101
BF = 10111111
74 = 01110100
04 = 00000100
00 = 00000000
XOR = 10001111 = 8F

This is the code that currently works fine with 6 byte messages:

void serialEvent() {
    
     if (Serial.available()>=6) { // there are bytes in the serial buffer to read
             IBUSbyte[0] = Serial.read(); // read in source byte.
             IBUSbyte[1] = Serial.read(); // read in length byte.
             LENGTH = IBUSbyte[1]; 
             if (IBUSbyte[0] == 0x50 || IBUSbyte[0] == 0x18 || IBUSbyte[0] == 0x68 || IBUSbyte[0] == 0xD7 || IBUSbyte[0] == 0x44 || IBUSbyte[0] == 0xC8 || IBUSbyte[0] == 0x80 && LENGTH <=7){   // ignore messages from some modules         
         for (int i = 2; i <= LENGTH+2; i++) {// read in rest of message.
             IBUSbyte[i] = Serial.read();
           }
             chkSUM(); // go off and see if checksum matches
           }
            
              else {
           debug();
           memset(IBUSbyte,0,sizeof(IBUSbyte));
           mySerial.println("No Match ");
           
      }
   }  
             
 } // End of IBUS read routine.

I've spent more hours than I care to remember trying to improve on this, but my tiny brain just can't figure it out. If anyone can suggest any improvements or a better way to deal with this, I'd appreciate it.

I was going to post the full code, but it made the message longer than allowed.

Here's my chkSUM code:

 void chkSUM() {
             checksumBYTE = 0;
             byteSTATE = 0;
         for (int i = 0; i < LENGTH+1; i++){
             checksumBYTE ^= IBUSbyte[i];} 
         if (IBUSbyte[LENGTH + 1] == checksumBYTE){
             byteSTATE = 1;}
         else if (IBUSbyte[LENGTH + 1] != checksumBYTE){
             byteSTATE = 0;}
              
         if  (byteSTATE == 1) {
              compare(); 
            }
         else if (byteSTATE == 0) {
             debug();
             checksumBYTE = 0;
             memset(IBUSbyte,0,sizeof(IBUSbyte));
             
      }
 }

Many thanks,

Ian.

1 Like

I haven't studied your code snippets but I have a few thoughts ...

Do you know the longest possible message?
Why not just receive all messages (regardless of length) and just ignore some of them?
How often are messages transmitted?

It's a good idea to write a short sketch that just focuses on the problem at hand. Then you can post the whole sketch here and get useful advice without lazy me having to wade through lots of stuff that you know how to do.

Even if you won't/can't write a short sketch post a complete sketch here (not just snippets) so we can see all of it.

...R

Hi Robin,

Robin2:
Do you know the longest possible message?

I don't know the theoretical maximum length, but the longest I've ever seen is 37 hex bytes long:

D0 23 3F A0 00 40 FE FE 00 00 00 00 08 AA 00 00 00 00 00 54 00 00 F9 00 00 00 00 00 00 00 00 00 00 00 00 00 23

Why not just receive all messages (regardless of length) and just ignore some of them?

I guess that's ultimately what I'm trying to do - just not very well. :blush:
I've figured out SPI, Rotary encoders, and all sorts of other good stuff, but simply reading in a string of serial data appears to be beyond me. Unfortunately, the whole project is dead in the water until I can figure it out.

How often are messages transmitted?

It varies, but there's quite a lot of traffic. Here's a small section from an IBUS log file with time stamps.

2014-04-14 17:07:55.015: D0 07 BF 5C FF FF FF 00 CB
2014-04-14 17:07:55.015: ! LCM --> GLO : Light dimmer, Data="FF FF FF 00"
2014-04-14 17:07:58.500: 68 03 18 01 72
2014-04-14 17:07:58.500: ! RAD --> CDC : Device status request
2014-04-14 17:07:58.515: 18 04 FF 02 00 E1
2014-04-14 17:07:58.515: ! CDC --> LOC : Device status ready,
2014-04-14 17:07:58.531: 18 0A 68 39 00 02 00 3F 00 07 01 78
2014-04-14 17:07:58.531: ! CDC --> RAD : CD_status, Stop Request=Pause CD=7 Track=1 CDs_Loaded=P1,P2,P3,P4,P5,P6
2014-04-14 17:08:04.468: E8 05 D0 59 50 00 34
2014-04-14 17:08:04.468: ! RLS --> LCM : Light control status, Control: Turn_lights_off Sensor_OK Level_Signal_invalid Reason:Data="50 00"
2014-04-14 17:08:08.031: 68 03 18 01 72
2014-04-14 17:08:08.031: ! RAD --> CDC : Device status request
2014-04-14 17:08:08.046: 18 04 FF 02 00 E1
2014-04-14 17:08:08.046: ! CDC --> LOC : Device status ready,
2014-04-14 17:08:08.046: 18 0A 68 39 00 02 00 3F 00 07 01 78
2014-04-14 17:08:08.046: ! CDC --> RAD : CD_status, Stop Request=Pause CD=7 Track=1 CDs_Loaded=P1,P2,P3,P4,P5,P6
2014-04-14 17:08:10.000: D0 07 BF 5C FF FF FF 00 CB
2014-04-14 17:08:10.000: ! LCM --> GLO : Light dimmer, Data="FF FF FF 00"
2014-04-14 17:08:14.484: E8 05 D0 59 50 00 34
2014-04-14 17:08:14.484: ! RLS --> LCM : Light control status, Control: Turn_lights_off Sensor_OK Level_Signal_invalid Reason:Data="50 00"
2014-04-14 17:08:17.546: 68 03 18 01 72
2014-04-14 17:08:17.546: ! RAD --> CDC : Device status request
2014-04-14 17:08:17.562: 18 04 FF 02 00 E1
2014-04-14 17:08:17.562: ! CDC --> LOC : Device status ready,
2014-04-14 17:08:17.593: 18 0A 68 39 00 02 00 3F 00 07 01 78
2014-04-14 17:08:17.593: ! CDC --> RAD : CD_status, Stop Request=Pause CD=7 Track=1 CDs_Loaded=P1,P2,P3,P4,P5,P6
2014-04-14 17:08:18.531: 68 05 18 38 03 00 4E
2014-04-14 17:08:18.531: ! RAD --> CDC : CD_control, Play

It's a good idea to write a short sketch that just focuses on the problem at hand. Then you can post the whole sketch here and get useful advice without lazy me having to wade through lots of stuff that you know how to do.

Even if you won't/can't write a short sketch post a complete sketch here (not just snippets) so we can see all of it.

Try this for size. It's the smallest I could make it while still remaining functional.

Here's some sample IBUS messages that it's happy with.
50 04 68 32 11 1F
50 04 68 32 10 1E

These are ignored, but the software handles it fine.
50 04 68 3B 02 05
50 04 68 3B 22 25

These will upset things.
44 05 BF 74 04 00 8E
44 05 BF 74 00 FF 75
C8 0F 80 23 42 32 42 6C 75 65 20 52 6F 6F 6D 20 15
D0 23 3F A0 00 40 FE FE 00 00 00 00 08 AA 00 00 00 00 00 54 00 00 F9 00 00 00 00 00 00 00 00 00 00 00 00 00 23

 #include <Wire.h>
  #include <SoftwareSerial.h>

 SoftwareSerial mySerial(7, 8); // 7 is Rx, 8 is Tx
 byte KEY_IN [7] = { 0x44 , 0x05 , 0xBF , 0x74 , 0x04 , 0x00 , 0x8E }; // Ignition key in
 byte KEY_OUT [7] = { 0x44 , 0x05 , 0xBF , 0x74 , 0x00 , 0xFF , 0x75 }; // Ignition key out
 byte VOL_UP [6] = { 0x50 , 0x04 , 0x68 , 0x32, 0x11 , 0x1F }; // Steering wheel Volume Up
 byte VOL_DOWN [6] = { 0x50 , 0x04 , 0x68 , 0x32, 0x10 , 0x1E }; // Steering wheel Volume Down
 byte LENGTH;
 byte checksumBYTE;
 byte IBUSbyte[20];
 boolean byteSTATE = false;
 int IBUSevent = 0; //boolean value set 1 if IBUS message detected
 int EventID = 0; // Event number (for case select)
 int CS_LWAKE = 4;
 
 void setup()
{
  Serial.begin(9600, SERIAL_8E1);
  mySerial.begin(115200);
  mySerial.println("--Digital Dock Volume Controller--");
  pinMode (CS_LWAKE, OUTPUT); // initialize pin.
  digitalWrite (CS_LWAKE, HIGH); // write pin high.
}

 void loop(){ 
  
   if (IBUSevent == 1){ 
    mySerial.println(); 
     
    switch (EventID){ 
      case 100:
        mySerial.println("UNKNOWN KEY");
        break;
       
       case 1:
        mySerial.println("Key IN");
        break; 
        
      case 2:
        mySerial.println("Key OUT");
        break; 
        
      case 3: 
         mySerial.println("VOLup");
        break;   
         
      case 4: 
        mySerial.println("VOLdown");
        break;         
    }
        IBUSevent = 0; 
    
  } 
        
 
 } // END OF MAIN LOOP
 
 void serialEvent() {
    
     if (Serial.available()>=6) { // there are 6 or more bytes in the serial buffer to read
             IBUSbyte[0] = Serial.read(); // read in source byte and store in IBUSbyte location 0
             IBUSbyte[1] = Serial.read(); // read in length byte and store in IBUSbyte location 1
             LENGTH = IBUSbyte[1]; 
             if (IBUSbyte[0] == 0x50 || IBUSbyte[0] == 0x44 && LENGTH <=7){   // ignore messages from some modules         
         for (int i = 2; i <= LENGTH+2; i++) {// read in rest of message.
             IBUSbyte[i] = Serial.read();
             }
               chkSUM(); // go off and see if checksum matches
             }
             else 
             {
               debug();
               mySerial.println("No Match ");
             }
   }  
             
 } // End of IBUS read routine.
 
 void compare(){
     EventID = 100;
     if (byteSTATE == 1) {
     if(memcmp(IBUSbyte, KEY_IN, 7) == 0 ) {EventID = 1;}
     if(memcmp(IBUSbyte, KEY_OUT, 7) == 0 ){EventID = 2;}
     if(memcmp(IBUSbyte, VOL_UP, 6) == 0 ){EventID = 3;}
     if(memcmp(IBUSbyte, VOL_DOWN, 6) == 0 ){EventID = 4;}
     IBUSevent=1; 
     } 
      else 
      { 
        IBUSevent=0; 
      } 
      
} 
      
 void chkSUM() {
             checksumBYTE = 0;
             byteSTATE = 0;
         for (int i = 0; i < LENGTH+1; i++){ // read in stored bytes and calculate XOR
             checksumBYTE ^= IBUSbyte[i];} 
         if (IBUSbyte[LENGTH + 1] == checksumBYTE){ // see if calculated XOR matches received XOR
             byteSTATE = 1;}
         else if (IBUSbyte[LENGTH + 1] != checksumBYTE){
             byteSTATE = 0;}
              
         if  (byteSTATE == 1) // if it matches, go and compare the received byte with the ones stored above
            {
              compare(); 
            }
         else if (byteSTATE == 0) // if it doesn't match, run debug() and display some diagnostic info
            {
             debug();
             checksumBYTE = 0;
             }
 }
 
 
 void debug(){
 
         mySerial.print("IBUSbyte = ");
         for (int i = 0; i <= LENGTH+1; i++) {         
              mySerial.print(IBUSbyte[i], HEX);
              mySerial.print(" ");
              }
              mySerial.println();
              mySerial.print("Source= ");
              mySerial.println(IBUSbyte[0], HEX);
              mySerial.print("Length= ");
              mySerial.println(IBUSbyte[1], HEX);
              mySerial.print("IBUS checksum = ");
              mySerial.println(checksumBYTE, HEX);
          if (IBUSbyte[LENGTH + 1] == checksumBYTE) {
              mySerial.println("IBUS checksum matches "); }
          else if (IBUSbyte[LENGTH + 1] != checksumBYTE){
              mySerial.println("IBUS checksum bad "); }
              mySerial.println();
              checksumBYTE = 0;
   
 }

It seems like you are reading the IBUS with hardware serial and looking at the results with software serial. Is that correct? Where is the software serial output appearing?

I only discovered serialEvent a few days ago and I suggest you don't use it as who knows when it decides to trigger itself.

If this were my project I think I would organize the program like this

void loop() {
  readIbus();  // a function that just reads in data and saves it in a buffer
  interpretData(); // a function that assesses what's in the buffer and sets a variable if something
                                needs to be done
  doSomethingWithTheData();  // a function that does stuff when necessary
}

You can easily test if there is something in the serial buffer with serial.available() without needing serialEvent to start the ball rolling.

Must you use 115200 baud for IBUS?

Do the IBUS messages have formal start and end characters? If so it will make reading much easier.

Rather than say more, and confuse matters, I will wait for your response.

...R

Robin2:
It seems like you are reading the IBUS with hardware serial and looking at the results with software serial. Is that correct? Where is the software serial output appearing?

Hi Robin,

I'm reading the IBUS with hardware serial because I need SERIAL_8E1 for the IBUS. I don't appear to be able to do that with software serial. IBUS communicates at 9600 baud.

Serial.begin(9600, SERIAL_8E1);

I'm only using software serial to view the results (in PuTTY) for debugging etc. This can be at any baud rate, 115200 is not a requirement. I won't need to use software serial at all once it's all working. Everything will be done by listening to the IBUS and acting on specific messages.

I only discovered serialEvent a few days ago and I suggest you don't use it as who knows when it decides to trigger itself.

So did I. It seemed like a nice way to call the IBUS read routine. I was doing it a different way before, but it's not caused me any problems so far.

If this were my project I think I would organize the program like this

void loop() {

readIbus();  // a function that just reads in data and saves it in a buffer
  interpretData(); // a function that assesses what's in the buffer and sets a variable if something
                                needs to be done
  doSomethingWithTheData();  // a function that does stuff when necessary
}




You can easily test if there is something in the serial buffer with serial.available() without needing serialEvent to start the ball rolling.

It's the first bit I'm struggling with. I've tried this sort of thing, but I just don't seem to get any data in my buffer.

int i = 0;
     while(Serial.available()>0){
       byte INbyte = Serial.read();
       IBUSbyte[i++] = INbyte;
      }

I'm probably being really thick, but I can't figure it out. Once it's in my buffer, I'm sure I can work with it.

Do the IBUS messages have formal start and end characters? If so it will make reading much easier.

Rather than say more, and confuse matters, I will wait for your response.

I'm afraid not. They just come in one after the other with no start or end characters.

Cheers,

Ian.

That serial code looks like it should work. How do you know it's not working? You haven't shown where the variables are declared so I can't say if you have an error there.

I would write a short sketch that just reads that and prints the buffer contents from time to time.

This would then become the function I called readIbus().

If I was using the code structure I suggested I would use IF (serial.available > 0) rather than WHILE so that the code doesn't sit reading characters at the slow baud rate (and sorry for getting my baud rates mixed up - a nice slow 9600 is best).

Have you an FTDI cable for software serial?

...R

Robin2:
That serial code looks like it should work. How do you know it's not working? You haven't shown where the variables are declared so I can't say if you have an error there.

I would write a short sketch that just reads that and prints the buffer contents from time to time.

This would then become the function I called readIbus().

Hi Robin,

I just took your advice and wrote a new sketch using the serial code that I posted above. I'm not sure what was going on in my original sketch when using this code, but in the new sketch, it works a treat. I think this is why I was so confused. It should have worked, but didn't.

If I was using the code structure I suggested I would use IF (serial.available > 0) rather than WHILE so that the code doesn't sit reading characters at the slow baud rate (and sorry for getting my baud rates mixed up - a nice slow 9600 is best).

I'll try that. Thanks.

Have you an FTDI cable for software serial?

Similar. It's something like this:

http://forum.arduino.cc/index.php?topic=66416.0

Thanks for your help :smiley:

Glad you are making progress.

The problem I have with serialEvent() is I don't know what happens if it is called again while you are working through the first call. There doesn't seem to any way to switch it off for a while. In any case it is unnecessary.

When I'm exploring something new I always try to work with the smallest possible piece of code so I can figure out what works and what doesn't. Later I can add the pieces together into a bigger project.

It's also much easier to get advice here with a small piece of code.

...R

Hi Robin,

this is still fighting with me. Should this work ?

#include <Wire.h>
 #include <SoftwareSerial.h>

 SoftwareSerial mySerial(7, 8); // 7 is Rx, 8 is Tx
 byte IBUSbyte[40];
 boolean IBUSprocess = false;
 
 

 void setup()
{
  Serial.begin(9600, SERIAL_8E1);
  mySerial.begin(115200);
  mySerial.println("--Digital Dock Volume Controller--");
  
  
}

void loop() {
  
  readIBUS();
  interpretData();
  processData();
  
}      

void readIBUS() {
       int i = 0;
     if (Serial.available()>0){
       byte INbyte = Serial.read();
       IBUSbyte[i++] = INbyte;
      }
}


void interpretData() {
  
     if (IBUSbyte[0] == 0x50 || IBUSbyte[0] == 0x44) {
       IBUSprocess = true;}
       
     else
        {
         IBUSprocess = false;
        }
}

void processData() {
  
 if (IBUSprocess == true) {
       
              mySerial.print(IBUSbyte[0], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[1], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[2], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[3], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[4], HEX);
              mySerial.print(" ");
              mySerial.print(IBUSbyte[5], HEX);
              mySerial.println(" ");
              }
  
      else
         {
         
         }   
}

If I send - 44 05 BF 74 00 FF 75 - all I get printed back is 44 00 00 00 00 00
if I send - 50 04 68 32 11 1F - - all I get printed back is 50 00 00 00 00 00

It only seems to be reading the first byte of the message and ignoring the rest. I'm not sure what's different between this and what I tried earlier, but this is the sort of issue I was having when I tried it in my original code.

Thanks again,

Ian.

A few thoughts come to mind ...

Is there any risk of receiving more than 40 chars - if so strange things will happen when ibusProcess and subsequent memory locations are over written.

I suspect there are two problems. First you have "int i = 0" in readIBUS() which means every character is saved in element 0. This value needs to be set to 0 only when you are ready to receive a new message. Second, it is printing the contents after a single character is received. It is probably receiving properly.

You need to organize things so that "IBUSprocess" is only true when a complete message is received - perhaps when a Cr or Lf is detected. There must be something that distinguishes one message from the next. If you don't know what it is I would just record 500 bytes and print them all out in Hex to look for a pattern.

...R

Edited at 12:40 to add the "first" problem.

Robin2:
A few thoughts come to mind ...

Is there any risk of receiving more than 40 chars - if so strange things will happen when ibusProcess and subsequent memory locations are over written.

I can't say for sure, but I don't think so. I can increase the array size if necessary.

I suspect there are two problems. First you have "int i = 0" in readIBUS() which means every character is saved in element 0. This value needs to be set to 0 only when you are ready to receive a new message. Second, it is printing the contents after a single character is received. It is probably receiving properly.

Good point. I was forgetting that readIBUS() only ran once. At the time (it was late) I was thinking the 'if' loop inside readIBUS() would run until no more serial was available, and then return to the main loop. Now corrected and working better.

You need to organize things so that "IBUSprocess" is only true when a complete message is received - perhaps when a Cr or Lf is detected. There must be something that distinguishes one message from the next. If you don't know what it is I would just record 500 bytes and print them all out in Hex to look for a pattern.

I think this is where I'm struggling. There really is no Cr or Lf. If I repeatedly send the same message ( 44 05 BF 74 00 FF 75 ), I just get a repeating string with no gaps, spaces or other characters.

445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75445BF740FF75

I believe the key is the length bit (second bit). Given this, you can calculate the length of the incoming message, but you need to receive this before you can act on it. This is what I was attempting to do in my original sketch, but couldn't get it flexible enough to cope with variable length messages.

Cheers,

Ian.

I believe the key is the length bit (second bit). Given this, you can calculate the length of the incoming message, but you need to receive this before you can act on it.

That looks like the answer - the second byte should allow you to control the read process. Trouble is though, how do you know that you're seeing the first byte of a message when you start? i.e. what if there is a message being transmitted on the bus when you power up the arduino? Are there periods of quiet on the bus? Are the source ids unique bytes that aren't used anywhere else in the protocol - I doubt it.

wildbill:
how do you know that you're seeing the first byte of a message when you start? i.e. what if there is a message being transmitted on the bus when you power up the arduino? Are there periods of quiet on the bus? Are the source ids unique bytes that aren't used anywhere else in the protocol - I doubt it.

All valid points. I think the hardware sorts out the power up side of things. I'm using a Microchip MCP2025 LIN BUS transceiver chip to convert the IBUS signals to TTL levels ( http://www.microchip.com/wwwproducts/Devices.aspx?dDocName=en558481 ). There are periods of quiet, but nothing of a fixed length as far as I can see. Source ID bytes are also used in the rest of the message, so that can't be relied upon either.

I think the answer is going to be a combination of the length byte and the checksum. Given both of these, you can tell if the message is a valid one.

Cheers,

Ian.

Looking at your sample data, I think I'd be inclined to throw serial data away at startup until the bus is quiet for some period. One hundred milliseconds would probably do it, but you can obviously experiment. You could use the same technique to sync up again if you're getting repeated checksum errors.

I believe the key is the length bit (second bit).

When I asked about marker bytes earlier you said there were none. It's a bit hard to give useful advice without the full picture.

What is the byte before the length byte? It doesn't seem to be constant. How do you know it is the first byte of a sequence and the byte following it will be a length byte.

How do you know the byte before the length byte isn't actually the last byte (perhaps a checksum) for the previous message?

...R

What is the byte before the length byte?

It's the source identifier. As far as I can tell, it's not an easy protocol to parse reliably. Apparently each source node is responsible for dealing with collisions too - not sure how. So if that collision detection is try & then back off, there may be a need to deal with garbage on the line too. Really need better docs on the protocol, but a brief google didn't turn up much other than the above.

I won't pretend to be an expert but if there is no specific way to identify the end of one message and the start of another I don't see how the data can be interpreted.

This is probably irrelevant, but, I note that the system uses a slightly unusual 8E1 serial system. I wonder if something is done with the serial bit stream that allows for the boundary between messages to be identified. It may also have something to do with detecting collisions (or the absence of them).
The regular Serial code to find the characters may not be able to "see" that stuff - it would just discard it as noise and wait for a valid character.
It might be interesting to write a DIY serial code (perhaps derived from SoftwareSerial) that would report all of the bit transitions as well as picking out the characters.

And, against all that, I have the impression from Reply #3 that @ian332isport is able to extract some of the messages - or perhaps it was just happy accident.

Before trying anything complicated I would like to see the hex values of, say, 100 consecutive bytes with no attempt to check any values.

...R

It looks to me as if there is some quiet time on the bus between messages so I think once you can detect a gap you can assume that the next character you see is the first in a packet and if the checksum is wrong you can discard and start the process again. So, I don't think that 100 hex bytes will tell you much unless you have time data for them too.

wildbill:
It looks to me as if there is some quiet time on the bus between messages

Just out of curiosity have you access to some additional information about the IBUS system? Do you have a link to it?

If there is quiet time between messages then it seems to me necessary to measure the time between the receipt of each byte - which should be fairly easy.

...R

Robin2:

I believe the key is the length bit (second bit).

When I asked about marker bytes earlier you said there were none. It's a bit hard to give useful advice without the full picture.

I explained the structure of an IBUS message in my first post.

IBUS protocol:

Example:

When key is inserted into the ignition lock these messages are sent.

44 05 BF 74 04 00 8F // in - key detected
44 05 BF 74 00 FF 75 // out - key removed

Structure of an IBUS packet (Ref first IBUS packet above):

1st byte is the Source ID - 0x44 - in this case the car immobilser module
2nd byte is length of the packet excluding the first two bytes (source & length) - 0x05
3rd byte is the destination ID - 0xBF - in this case the light control module
4th byte onwards is the actual data - 0x74, 0x00, 0xFF
Last byte is the checksum - 0x8F

Checksum is generated by XOR of the entire packet excluding the checksum itself.

HEX - Binary
44 = 01000100
05 = 00000101
BF = 10111111
74 = 01110100
04 = 00000100
00 = 00000000
XOR = 10001111 = 8F

What is the byte before the length byte? It doesn't seem to be constant.

As above, it's the source ID.

How do you know it is the first byte of a sequence and the byte following it will be a length byte.

To be honest, I'm not sure. In practice, this doesn't seem to happen. I guess it may be handled by the IBUS trasceiver IC, but the datasheet doesn't give much indication.

How do you know the byte before the length byte isn't actually the last byte (perhaps a checksum) for the previous message?

The structure of an IBUS message is fixed. The byte before the length byte will always be the source ID

have the impression from Reply #3 that @ian332isport is able to extract some of the messages

I'm using a piece of software called Navcoder to log the IBUS in the car. http://www.navcoder.com/
This allows me to log all the traffic on the IBUS as it happens and log the results. It also allows me to transmit messages on the bus which allows me to debug the software on the bench without dragging everything out to the car. I only go to the car for final testing. This is what highlighted the issue with long messages.

Before trying anything complicated I would like to see the hex values of, say, 100 consecutive bytes with no attempt to check any values.

I'll see if I can get that later today.

Just out of curiosity have you access to some additional information about the IBUS system? Do you have a link to it?

Have a read of this (the first part at least).

http://web.comhem.se/mulle2/IBUSInsideDRAFTREV5.pdf

Thanks for your continued interest/support.

Ian.