Struggling with my BMW IBUS projects. Serial.read related issues.

Thanks, that link is helpful - even if it doesn't have all the information.

Sorry, I had not spotted the existence of the message length in your first post because I was focusing on how to distinguish one message from the next - which we haven't yet figured out.

Hopefully the stream of 100 bytes or so will throw up something useful.

The IBUS standard seems to use RTS and CTS for collision detection. I don't know if the Arduino serial interface can report the values of those lines. They might be a signal for the transition from one message to the next. Hopefully that won't be necessary.

...R

I think more through luck than anything else, but I appear to have it working. :smiley:

It’s possibly not the most elegant solution, but I can’t get it to fail when connected to the car.

void ReadIBUSmsg()
 { 
   boolean received = 0;
   byte IBUSbyte[40];
   int len = 0;

   while(Serial.available() > 0 && received == 0 )
   { 
     IBUSbyte[0] = Serial.read();  // Read source byte
     while (received == 0)
     {
       if(Serial.available() > 0)
       {
         IBUSbyte[1] = Serial.read();  // Read length byte
         received = 1;       
       }
     }

     received == 0;
     len = IBUSbyte[1] + 1;
     LENGTH = IBUSbyte[1];
     int i = 2;
     i = 2;
     while(i <= len) // Read remainder of message
     {
       if(Serial.available() > 0)
       {
         IBUSbyte[i++] = Serial.read(); 
       }
     }

I would love to know WHY it is working.

Did you collect the sample of 100 consecutive bytes? I would love to see them.

...R

Robin2:
I would love to know WHY it is working.

Did you collect the sample of 100 consecutive bytes? I would love to see them.

…R

I would also be interested to understand why it works. It is still working though.

These logs were collected with the following code.

#include <Wire.h>
 #include <SoftwareSerial.h>

 SoftwareSerial mySerial(7, 8); // 7 is Rx, 8 is Tx
 byte IBUSbyte[40];
 const int CS_LWAKE = 4;
 

 void setup()
{
  Serial.begin(9600, SERIAL_8E1);
  mySerial.begin(115200);
  mySerial.println("--IBUS read test--");
  pinMode (CS_LWAKE, OUTPUT); // initialize pin.
  digitalWrite (CS_LWAKE, HIGH); // write pin high.
  
}

void loop() {
       
  readIBUS();
}      

void readIBUS() {
       if (Serial.available()>0){
       byte INbyte = Serial.read();
       mySerial.print(INbyte, HEX);
      }
}

https://dl.dropboxusercontent.com/u/100050743/NavCoder_Log_20140421.log
https://dl.dropboxusercontent.com/u/100050743/navcoder%20log.txt

I wonder why "Navcoder" appears in the file names? If you are feeding the data into a Navcoder program I suspect it is "interpreting" the data and we are not seeing the "raw" data.

Bytes with the value 0 seem to be missing from the .txt file.

Is the data in the .txt file supposed to be identical to that in the .log file or were the two collected on separate occasions.

Is the data being collected directly from the car?

If you can just view the data in the Serial Monitor it should include every byte. You could then copy and paste a piece of it here.

Interesting ...

...R

Hi Robin,

In theory at least, the two files should show the same data. One file is simply a cut and paste from the PuTTY terminal window into notepad. This should be the raw data that my Arduino sketch is capturing. The other file is the Navcoder log showing the same data with time stamps etc. Navcoder names its own log files, and I had to name the Notepad file.

You will find the .txt file does show zeros, but not the leading ones. Although annoying, it appears normal that leading zeros are not shown when you receive HEX bytes. 0F will just show as F, 00 will show as 0. I've yet to find a way to display the leading zeros in the terminal window.

All data was taken directly from the car.

Ian.

ian332isport:
You will find the .txt file does show zeros, but not the leading ones. Although annoying, it appears normal that leading zeros are not shown when you receive HEX bytes. 0F will just show as F, 00 will show as 0. I’ve yet to find a way to display the leading zeros in the terminal window.

Thanks. I hadn’t thought of that. In my own programs I usually print a SPACE between each byte so I probably never noticed. I’ll have another look and see if I can make sense of the data. I wonder will it be possible to know where to add the missing 0s?

It would be great if you could collect some more data with the bytes visually separated from each other.

…R

Edit to add …

I have had another look at the data.

As far as I can see it is impossible to replace the missing 0s. It is easy to figure that a 0 should be 00 but it is impossible to know whether 3B2 should be 03 0B 02 or 3B 02 or anything else.

And I’m pretty sure your two files don’t have the same data.
…R

Edit to add more …

Google led me to this web page with the following statement

Unlike most similar single wire network protocols, the I-Bus has no apparent way to identify the start of a packet, so this must accomplished in software by looking for a delay between messages.

…R

Hi, this is what you are doing wrong: You seem to forget that ibus is very slow. 9600 means roughly one byte per milisecond. When using theif (serial.available() >0) loop, after the mcu reads the first byte, there is nothing left in the buffer, so it exits the loop, or worse it uses 0 in the second for loop. Same problem when you were using if (serial.available() >=6) and you could not read the 7th byte. You need to do something like this:

void serialEvent(){
  while (Serial.available()) {
  read sender byte
  check to see it is a valid sender 
  wait for the next to come in. 
  read length byte
  wait
...etc}

your solution works because you lock into the while loop with the received == 0 condition...

ian332isport:
I’ve yet to find a way to display the leading zeros in the terminal window.

you can try this:

   for (byte j = 0; j <= ibusMsgSize; j++) {
    if (ibusMsg[j] < 16) Serial.print('0');
    Serial.print(ibusMsg[j],HEX);
    Serial.print(" ");
  }

iqueban: Hi, this is what you are doing wrong:

Did you read the quote in my Reply #27?

What is really needed is code that measures the gap between bytes being received so as to find gaps between transmissions (presumably gaps are 2 or more times the length of the inter-byte gap in a message). Then take in all the bytes until the next gap. Then analyze the catch.

It's all a bit pointless if the OP is already satisfied.

...R

Hi all,

I'm doing a similar project albeit with a different microcontroller. Anyway I have the information you're looking for... the ibus messages are separated by a pause of minimum 11 milliseconds (i.e. the bus is idle for at least 11 msec) and it should be well enough considering that at 9600bps a bit takes 0.104 msec. So by setting a timer to detect this bus idle time you can detect the beginning of a new message if a char appears after that period. I'll follow this as maybe in the future I'll move on to arduino myself, but in the meantime good luck with the project.

Thanks @xrwz, that's useful information.

Do you have a link to a document that explains that. I didn't see it in the few places I looked.

...R

Hi,

Sorry to have vanished from this thread. I had to drop everything and deal with some family issues.

Back on track now though :D

Very interesting info regarding the gap between messages. Although my current sketch is working for the most part, I don't feel like it's as robust as it could be. On the odd occasion it does seem to get out of sync and cause problems.

After more reading about serial communication on the Arduino, it looks like the start, stop and parity bits are being stripped out by the Arduino software and only passing the actual 8 bits of data to the serial buffer. I think you would need to do some sort of port level programming to get this data.

I'm going to try and re-write the IBUS read part of the sketch and try and work with the 11 msec delay between messages. This seems like a far better way of capturing messages and discarding them if they're either corrupt, or just not one I want to process.

I now need to figure out how to measure the idle time between messages....

Thanks,

Ian.

I've taken delivery of a logic analyzer today, so I can hopefully see exactly what sort of gap I'm getting between messages and what is [u]actually[/u] being sent.

I'll update when I have more info.

In the mean time, I don't suppose anyone can suggest how best to measure the idle time between messages ?

I think I can use bitRead(PIND,0) to monitor the state of the Rx pin, but not sure if I need to use a timer or interrupt or some other mechanism to detect a bus idle time of say 6 milliseconds (or whatever idle time is suitable).

For now I'll keep reading the Arduino Cookbook and see what I can figure out.

Thanks,

Ian.

You should be able to do something with the bus read routine. Whenever you check serial.available, if there's nothing there, check a flag variable. If it indicates that no data is a new condition, note the time. Whenever you do read a character, reset the flag. Use the flag & time to tell you if the 11 ms of quiet on the bus have elapsed.

@Riva, you have been very kind to the documentation for serialEvent().

In my view it has no place in any application.

It will be easy to write some code that consumes all the available characters using Serial.read() and then keep track of the time interval until the next character arrives.

...R

Thanks to everyone for your continued input.

I still need to hook it up to the car to see what sort of gaps I'm getting in the real world, but the logic analyser is working very nicely on my simulator software. This is what a single (Volume Up) message looks like.

I'll report back once I've got some real data.

Thanks,

Ian.

Right, I've hooked the logic analyser up to the car and have the results.

Firstly, the time between messages varies a huge amount, and is in most cases well under the 11 ms suggested previously.

I've not checked every single message transmitted, but the shortest time I've seen between messages is 1.91 ms. When you look at the length of an 0xFF byte (the byte with the longest bus high duration), it's 0.830 ms. From this we can say that any bus high time greater than 0.830 ms is effectively bus idle time between messages. For reference, an 0x00 byte is low for 1.039 ms which I believe is the correct pulse width for 9600 baud.

So now I need to figure out how to trigger a timer on a rising edge and flag the end of the message if the time is greater than 0.83 ms. I guess the timer can stop on the next falling edge, or just reset when the 'message ended' flag is set.

Ian.

ian332isport: So now I need to figure out how to trigger a timer on a rising edge and flag the end of the message if the time is greater than 0.83 ms. I guess the timer can stop on the next falling edge, or just reset when the 'message ended' flag is set.

I'm not sure if it makes any logical difference, but what you need to detect is the start of a message - i.e. the first bit after the "long" interval. You can calculate the end because it gives you the number of bytes - provided you have correctly started interpreting bits at the start of the message.

If you have a value that increments and which is reset by an interrupt. If the value is above some threshold you will know that there has been a long interval. This could go on all the time and just be ignored except when needed so that the same code could do this job as well as other stuff.

...R

Hi Robin,

I can't decide if it's better to read in the entire message using the idle time at the end of the message, or detect the start of the message and then use the length byte to read in the required number of bytes as you suggested above. To be honest, it's all a bit academic at the moment, as I can't figure out how to measure the idle time.

I guess I'll just have to keep plugging away at it until I can find a solution.

Thanks,

Ian.