Unexpected Data from Serial Connection

Hi All,

I have been on here a few times recently and its been really useful - so I'm back again with my latest issue!

I am trying to connect to a standalone automotive ECU in order to stream some live data from it in order to make a simple dash. The ECU connection is made via an RS232 cable, and I'm using an Arduino Uno with a MAX232 shield.

The ECU firstly needs an initial request in order to start streaming data, but will then need tester present ACks to be sent cyclically every 1-2s. The ECU should respond to the initial request without an ACKs though obviously.

The ECU connection is set to a baudrate of 230400 as specified by the manufacturers (earlier ECU FW was 112500, but I've checked and mine is definitely the later FW requiring 230400).

So presently, I can send the request to the ECU, and I see the TX led on the MAX232 flash. I then see about 2s worth of flashing on the RX led, showing I'm receiving data which is great. However the data im recieving when printed to the serial monitor is incorrect.

I am seeing every time I connect:

Received:
4E 85 31 A0 8 2 0 CD 80 20 2 10 0 1 21 0 8 1 44 40 83 80 1 2 40 4 4 1C C0 46 (only part of the message exemplified here)

But what I should see according to the ECU documentation and what I see when I use the manufacturers tuning sw is:

Received:
4D 45 47 01 0F 00 02 00 6C 00 03 00 04 11 00 02 12 00 00 0F 00 02 10 00 01 (only part of the message exemplified here)

The 0x4D and 0x45 are key - they appear in every request and response and are used as sync bytes. So at the minute Im not sure exactly what is happening. If anyone could offer any suggestions that would be much appreciated!

Ive attached the code below, there isnt anything there at the minute to deal with sorting the received data, I just want to get the correct initial response first, and deal with the frame processing after:

#include <SoftwareSerial.h>


#define DEBUG                             (1)

/**************************************************/
/*      ECU Comms                                 */
/**************************************************/
#define MIN_FRAME_LEN                     (10)
#define MAX_FRAME_LEN                     (768)
#define SYNC_CH_0                         ('M')
#define SYNC_CH_1                         ('E')

#define MSG_TYPE_REQ                      (0x0)
#define MSG_TYPE_RSP                      (0xF)

#define CLASS_ID_REPORT                   (0x0)

#define MSG_ID_REPORT                     (0x0)
#define MSG_ID_ACK                        (0x1)
#define MSG_ID_SET_STATE                  (0x2)

byte rx_buf[MAX_FRAME_LEN];
int rx_idx;
unsigned int rx_frame_len;

struct proto_frame {
  byte sync_ch[2];
  unsigned int len;
  byte type;
  byte class_id;
  byte msg_id;
};


SoftwareSerial Comms_serial(8, 9); //232_TX,232_RX // Config to assign Digital io ports for the RS232 Shield

struct proto_frame rx_frame;



/* This is to create and send a request to the Me221, in order to get a response back. This will be sent once,
  and for persistent comms need to introduce a cyclical Ack sent every 1s*/
void start_reporting_req() {
  byte req [] = { 0x4D, 0x45, 0x01, 0x00, 0x00, 0x00, 0x02, 0x01, 0x03, 0x05,   };

  Serial.print("Sending: ");
  Comms_serial.write(req, sizeof(req));
  Serial.println();
}


void setup() {
  /* Init debug serial for serial monitor*/
  Serial.begin(9600);
  Serial.println();
  Serial.println("Serial debug line open");

  /* Init ECU comms - check baudRate, its dependant on Me221 FW version:
    115200 baud rate (for versions <2.1.2.)
    230400 baud rate (for versions >=2.1.2)  */

  Comms_serial.begin(230400);
  rx_idx = 0;
  delay(2000);
  Serial.println("Initialized software serial");
  start_reporting_req();
}



/* This is to chop and decipher the frame reported back to us. Each frame needs to be segmented in order to dechiper the
  data we rhave received from the ECU*/

/* Main loop that handles the buffered response byte by byte and checks for sync bytes (M E) to signify the start of the frame.
  If successful message received, its sent for processing*/

void loop() {
  while (Comms_serial.available() && rx_idx < MAX_FRAME_LEN) {
    rx_buf[rx_idx++] = Comms_serial.read();

  }

  Serial.print("Received ");
  Serial.println(rx_idx);

  for (int i = 0; i < (rx_idx - 1); i++) {

#if (DEBUG == 1)
    Serial.print( rx_buf[i], HEX);
    Serial.print(" ");
#endif


  }    
  Serial.println("\n");
}

Software serial will struggle above 19200 baud on an UNO.

For the baud rate you need, you definitely need a hardware serial port. You should check that the UART clock can generate that baud rate as well.

Thanks for the response! I assume I will need something other than an UNO to handle this from what you are saying? Do you have any recommendations - sorry Im completely new to arduino!

It seems that you can swap hardware and Software Serials, connecting the ECU to hardware one and using the SoftwareSerial to output the diagnostics to the console.
In that case you can still use the Uno and just will need a USB-UART transmitter to connect the sowtware serial pins to the USB of your PC

Try the suggestion above. A lot may depend on the quantity of data being received. The UNO is only a 16mhz device so could get swamped.

Thanks both, I guess if I need to buy more hardware either way, then looking at other feedback it would be sensible to purchase a board with more hardware serial connections, something like a Mega perhaps? In terms of data, there could be up to 769 bytes received every 1-2s so not insignificant.

If I understand correctly then, I can just keep the existing USB connection n the Mega for the serial Arduino Serial Monitor, and then assign one of the additional hardware serial connections to the MAX232 -> ECU connection?

Thanks again for the help!

1 Like

Yes, that's correct, but there are other Arduino options that don't involve the huge physical footprint of the Mega, but do have more than one hardware serial. Think about what features you'll want, then list them and someone can point at an optimal solution. If all you need is 2 hardware serial, why use something with 50+ unused IO?

That will get you 4 serial ports but its still only a 16mhz setup.

To make it easier to search for @scortedvan , a few examples:

  • Pro Micro has 2 serials
  • Nano Every has 4 serials
  • Rpi2040 has 2 serials
    .... also ESP8266, ESP32 and STM32 based "arduinos" has more than 1 serial

Again, thanks everyone for their thoughts, really helpful. Great to see such an active forum!

I think looking at some of the options listed, the Nano Every appears to fit the bill. Its physical size will lend itself to in car fitment, and the additional hardware serial port will cover what I need.

To add some more detail, I am building a (basic!) digital dash for a car. The intention is to use a GPS sensor and then display speed - I have this working already using softwareSerial and this will remain a constant display on some sort of screen - OLED would be a preference there I think although it won't be huge, perhaps 6"x3"?

Then on another area of the screen I will be able to cycle through various ECU inputs such as RPM, Coolant temp, air temp etc. This is where I need the connection to the ECU which as above requires a MAX232 and a baud rate of 230400.

Finally the ability to connect my laptop to the USB connection on the Nano for debugging in the serial monitor will help massively.

If anyone has any thoughts or comments then please feel free to pass them on.

With displays as well, maybe one of the Adafruit feather boards or one of the Teensy boards. I have no experience of either, but they have more RAM and a higher clock speed.

Thanks for the suggestions, they both look like suitable options aswell.

Interestingly I actually found a USB adapter kicking around from an old project, so I have tried using the hardwareSerial connections for the ECU, and softwareSerial for the PC connection. With this setup I am receiving data from the ECU as before, and at first I thought it was correct but sadly not:

Received
4D 51 24 0F 2 B5 3 84 44 82 22 0 8 2 4 1 43 2 8 1 11 0 4 0 46 0 23 80 22 82 45 2 26 4 45 20 16 20 19 20 27 A0 28 A0 3A 20 3B 20 10 12 4 4F 20 4B 20 4C 20 54 20 55 20 45 20 46 20 47 20 49 20 C8 20 20 4D 20 4E 20 50 20 51 20 4F 20 52 20 53 20 56 20 D7 20 (only part of the message exemplified here)

And again it should look like this , with the first two bytes being 0x4D, 0x45

Received:
4D 45 47 01 0F 00 02 00 6C 00 03 00 04 11 00 02 12 00 00 0F 00 02 10 00 01 (only part of the message exemplified here)

Would it be safe to say at this stage I am at the limitation of the hardware, and I would need something with more processing power to achieve this? Or should 16mhz be enough to churn through some reasonably simple code but receive 769 bytes every 1-2s ?

If so, the boards mentioned by @markd833 seem to offer the required serial connections and more processing so I'm happy to purchase one unless there are any other boards I should consider?

Kind Regards

Joe

Please show your code after swapping the ports.

Yeah sure, here is the revised code:

#include <SoftwareSerial.h>

#define DEBUG                             (1)

/**************************************************/
/*      ECU Comms                                 */
/**************************************************/
#define MIN_FRAME_LEN                     (10)
#define MAX_FRAME_LEN                     (768)
#define SYNC_CH_0                         ('M')
#define SYNC_CH_1                         ('E')

#define MSG_TYPE_REQ                      (0x0)
#define MSG_TYPE_RSP                      (0xF)

#define CLASS_ID_REPORT                   (0x0)

#define MSG_ID_REPORT                     (0x0)
#define MSG_ID_ACK                        (0x1)
#define MSG_ID_SET_STATE                  (0x2)

byte rx_buf[MAX_FRAME_LEN];
int rx_idx;
unsigned int rx_frame_len;

struct proto_frame {
  byte sync_ch[2];
  unsigned int len;
  byte type;
  byte class_id;
  byte msg_id;
};

SoftwareSerial serialMonitor (8, 9); //232_TX,232_RX // Config to assign Digital io ports for the RS232 Shield

struct proto_frame rx_frame;

/* This is to create and send a request to the Me221, in order to get a response back. This will be sent once,
  and for persistent comms need to introduce a cyclical Ack sent every 1s*/
void start_reporting_req() {
  byte req [] = { 0x4D, 0x45, 0x01, 0x00, 0x00, 0x00, 0x02, 0x01, 0x03, 0x05,   };

  serialMonitor.print("Sending: ");
  Serial.write(req, sizeof(req));
  serialMonitor.println();
}


void setup() {
  /* Init debug serial for serial monitor*/
  Serial.begin(230400);
  serialMonitor.begin(115200);

  serialMonitor.println();
  serialMonitor.println("Serial debug line open");

  /* Init ECU comms - check baudRate, its dependant on Me221 FW version:
    115200 baud rate (for versions <2.1.2.)
    230400 baud rate (for versions >=2.1.2)  */

  rx_idx = 0;
  delay(2000);
  serialMonitor.println("Initialized software serial");
  start_reporting_req();
}

/* Main loop that handles the buffered response byte by byte and checks for sync bytes (M E) to signify the start of the frame.
  If succesfull message recieved, its sent for processing*/

void loop() {
  while (Serial.available() && rx_idx < MAX_FRAME_LEN) {
    rx_buf[rx_idx++] = Serial.read();

  }

  serialMonitor.print("Received ");
  serialMonitor.println(rx_idx);

  for (int i = 0; i < (rx_idx - 1); i++) {

#if (DEBUG == 1)
    serialMonitor.print(rx_buf[i], HEX);
    serialMonitor.print(" ");
#endif

  }

  serialMonitor.println("\n");
}

You need to reduce this baud rate. Try 9600 or 19200 on a software serial port. I wouldn't go any higher with a 16MHz UNO.

Do you have a genuine UNO or a clone? If it's a clone, is it using a crystal to generate the 328P clock or a resonator. I think you should be ok with a crystal but a resonator may not be accurate enough.

Thanks @markd833 , I did try running with different baud rates, including 9600 but it had no effect on the data I was able to log.

In terms of the hardware I am using, its not a genuine Arduino Uno, its an ELEGOO UNO R3 and has a 16MHz crystal installed on the board

Ok, another thought. You have an RS232 to TTL serial adapter and a TTL serial to USB adapter. You should be able to join those 2 together to create an RS232 to USB adapter.

I would try and use that with a PC terminal program to see if you can receive the expected data stream on a PC. That should give you confidence that the data is as expected and the baud rate is 230400.

Do you have any test kit available - maybe one of those $10 logic analysers?

Thanks for the suggestion @markd833, that's a really neat idea!

So I have fired up HTerm and connected the RS232 and USB adapter together setting the baud rate to the required 230400. I fired the request sequence in, and hey presto - I received exactly the response I expected!

So at this stage, I know some things for sure then:

  1. The baudrate required for the ECU most certainly is 230400, exactly as per the spec sheet suggests.

  2. I don't have an odd issue with the MAX232 adapter I'm using, nor the USB adapter.

  3. The request I am sending is correct (although this was expected as I was always receiving data after sending the request)

And I think that really only leaves as potential issues, my code or the arduino itself? In terms of the code, I'm inclined to think that should be ok - as I do actually transmit and receive something. So im swinging towards the Arduino?

With regards to test kit, I don't have anything useful at home unfortunately, its all at work. Best I have here is a multimeter!

It does point to an issue with your UNO. I would probably start to consider the main crystal frequency. If it's a cheap crystal and slightly off frequency, then the hardware UART will probably struggle to acquire the byte.

Here's a table from the 328P datasheet showing the percentage error in baud rates assuming a 16MHz clock:

At 230400 baud you could potentially have an 8.5% error. You would need to look at the arduino library code for the UNO hardware serial port to see if the U2Xn (double speed) bit is set or clear in the USART control and status register A.

Hmm ok thanks - I may be reaching the limit of my abilities right now I'm afraid. Im looking at the HardwareSerial.cpp file but I don't see specifically a point at which U2Xn is set? I'm sure it is, it is just above me! Are you suggesting there is something in the code I could adjust to make this work, or am I just looking for confirmation that the board is indeed the issue?

At this stage looking for something with better processing capabilities does sound like it may be required right?

Edit: Ive looked a bit closer and actually I do see the setting, it defaults to try double speed mode, unless the baudrate is specifically 57600 or lower than 4095. So in short, yes I am using double speed mode presently!