Searching and parsing data from serial.read

Thanks again to all that have responded - I now understand the string question and the line feed in the monitor worked as well. Command + t also smartened up the formatting.

Good - so now it works as you want, right?

I was trying to understand what you say there but you got me lost..

Just to re-iterate the message format - depending on the engine parameter, the packet can consist of a an address byte followed by one data byte, or an address byte followed by two data bytes. The first byte is always the engine identity which defaults to 80 (hex), the packets follow that. Engine parameters below 192 contain 1 data byte, 192 and above contain 2 data bytes.The exception is if the second byte is FF which indicates a second page of parameters, in this case the first address byte is the 3rd byte in the stream rather than the 2nd. Finally, if the address byte is 192 I want to dump the rest of the line as it contains trouble codes, which is not what I'm after.

So I went to check the SAE J1587 protocol on the internet

Turns out it's a bit more complicated than you describe but still probably relatively easy to parse on the fly with a state machine (as long as you don't receive PID 0xFE because then it seems things are manufacturer dependent).

There is also a notion of a special command RTS that will be sent to explain what data will follow.

To summarize the description:

The first byte of a message contains a message identification (MID) which is node specific. J1587 defines MIDs in the interval 0x80-0xFF.

The first byte after the MID is a parameter identification (PID). A PID is (usually) one byte long and can contain values 0x00 to 0xFF.

Every PID is followed by a number of parameter data bytes. Their number and interpretation depend on the value of the PID.

A message can contain several PIDs next to each other.

  • PIDs from 0-127 (and 256-383) describe data parameters that are one byte long
  • PIDs from 128-191 (and 384-447) describe data parameters that are two bytes long
  • PIDs from PIDs 192-253 (or 448-509) describe data parameters that have more than two bytes. The first byte following these PIDs will contain the number of data parameter bytes
  • PID 0xFE is used for transmission of special commands, data and information intended for a certain node on the bus. Data parameters sent after this PID can be determined by the manufacturer of a device.
  • PID 0xFF is used for extension of a PID to two bytes, that is to say that the following byte also is a PID. With this extra PID values up to 511 can be used. If the first PID is 0xFF the following PID is interpreted as modulo 256 (0=256, 1=257).

The message ends with a one byte long checksum (two’s complement to the sum of all data bytes in the message) if the sum modulo 256 of all bytes in a message, including the checksum, is zero if the message is valid.

Other info

  • Data is sent with the least significant byte first.
  • Alphanumerical data are sent with most significant byte first and is interpreted according to the ISO Latin 1 (8859-1) standard.
  • Signed integers are sent as two’s complement.
  • All temperatures are in degree Fahrenheit.
  • Floating point numbers adhere to the IEEE floating-point standards.

the J1587 standard defines the MIDs. For example 0x80 is the engine

a J1587 message length is limited to 21 bytes but Connection Oriented Transport Service provides an way to send more than 21 bytes long messages and can handle messages segmented into blocks consisting of 15 bytes each (except last one which can be less).

To transfer segmented messages between two nodes a request of a virtual connection has to be sent from transmitter to receiver. This is done with the RTS command which contains the number of segments and the total number of bytes to be transmitted. The receiver has to accept the request by sending a CTS command which contains the number of segments it can receive and which segment to send first. When this handshaking has been successfully performed data can be transmitted with the connection mode data transfer PID.

A message has up to 3825 bytes. Every segment is given a header with a segment number and then sent in a J1708 message with a MID and a special PID. The receiver of the segmented message removes the header and checksum and reconstructs the original message from the remaining 15 data bytes.

The service for transmitting segmented messages uses two special PIDs:

  • Connection management PID (0xC5 = 197) controls the start and end of a connection, flow control and receipt of a message.
  • Connection mode data transfer PID (0xC6 = 198) is used exclusively for data transfer