reading serial data *without* it being converted to ASCII??

I'm hoping there is an easy solution to this that I have just not been able to find in my searches.

I'm trying to read a stream of data into a serial port on the Arduino (mega 2560). It is a repeating hex string with a header (fe ff fe). kinda like this:

fe	ff	fe	00	00	00	45	00	45	00	45	00	45	00	45	00	45	00	46
00	46	00	46	00	46	00	46	00	46	ff	f9	00	3c	00	00	00	00	00
84	00	00	4d	ce	00	00	00	3b	63	00	20	ff	de	00	90	00	69	00
3b	15	b4	01	1f	00	0	00	00	00	00	00	00	00	6c	c6

If I read this via a serial/usb converter to a serial monitor program, the data is read as I expect it to.

The problem, it seems, is that the serial data read with an Arduino is converted to ASCII. Is there a way to read this stream of data not as ascii, or convert it back to it's original form after reading?

For testing purposed, I've tried to break the program down to it's simplest form. I've tried many combinations of defining the 'incomingByte' and Serial.print/Serial.write/Serial.readBytes..etc. that I can think of. Here is an example.

uint8_t incomingByte = 0; // for incoming serial data
int i=0;

void setup() {
  Serial.begin(9600); //  used to write to serial monitor
  Serial1.begin(9600); // used to read stream of data 
  Serial2.begin(9600);  // used to write to com port and viewed with serial port monitoring program (not the arduino serial monitor) 
}

void loop() {

  for (i = 0; i < 150; i++)     // want to loop this 100 times to capture 2 cycles of data 
  {
    // send data only when you receive data:
    if (Serial1.available() > 150)
    {
      // read the incoming byte:
      incomingByte = Serial1.read();       // read a byte
  
      Serial2.print(incomingByte);   // print to serial 
      Serial2.print(" ");               // just adding a space
    }
  }
Serial.println("end \n");               // end of 100 characters and new line

  
}

(there is probably a similar problem when doing a serial.print or serial.write, that I suppose will be the next question, or possible answered with this first one)

I'm an amateur, trying to learn things, so please be patient with me. Thanks in advance.
(please don't link me to the "serial input basics" thread. I've read it.)

-Jeff

Where is the data coming from ?

jcaplins:
I'm hoping there is an easy solution to this that I have just not been able to find in my searches.

please don't link me to the "serial input basics" thread. I've read it.

Gee, I can't wait to see how this thread turns out...

jcaplins:
The problem, it seems, is that the serial data read with an Arduino is converted to ASCII.

Serila.read() just reads a byte of data, its carries out no 'conversion' on that byte, nor should it.

You can then print that received byte how you like, as a decimal number, hexadecimal, binary or indeed as an ASCII character.

jcaplins:
(please don't link me to the "serial input basics" thread. I've read it.)

-Jeff

That's good to know. It will make it very much easier to help you if you run the Serial Input Basics program (the 2nd example will probably be most suitable) and post here the output that it produces.

...R

if (Serial1.available() > 150) How big is the serial input buffer?

This should be the basics for testing; it writes the received byte as received without conversions.

void loop()
{
  if (Serial1.available() > 0)
  {
    byte incomingByte = Serial1.read();
    Serial2.write(incomingByte);
  }
}

Not tested.

Is each block of data a fixed length, or does it have a length indicator in the data or a specific terminator?

As the previous post shows, you want to use Serial.write() to send the received byte to your monitor program, Serial.print() will produce the ascii representation.

The serial input basics thread is applicable to this, but it does take a bit of modification to look for a 3-byte sequence as the start indicator, and without knowing more about the specific data format its hard to tell how to recognize the end of data.

The bytes get sent in decimal ASCII because you call "Serial2.print(incomingByte);" and not "Serial2.write(incomingByte);".

UKHeliBob:
Where is the data coming from ?

An engine monitoring device that reads a bunch of sensors and displays them on a small screen. The data is also sent out via serial. I can't change anything about this device.

Nick_Pyner:
Gee, I can't wait to see how this thread turns out...

One can dream, can't we?
I've read the input basics thread... I didn't see anything that would help me. I will read it again.

johnwasser:
The bytes get sent in decimal ASCII because you call "Serial2.print(incomingByte);" and not "Serial2.write(incomingByte);".

I have tried both ways. different number, but still incorrect.

david_2018:
Is each block of data a fixed length, or does it have a length indicator in the data or a specific terminator?

I would say fixed length. It's 8 bit, no parity, 1 stop bit, with the unique header that can not be repeated anywhere in the data. The last byte of the stream is a checksum.

david_2018:
The serial input basics thread is applicable to this, but it does take a bit of modification to look for a 3-byte sequence as the start indicator, and without knowing more about the specific data format its hard to tell how to recognize the end of data.

I'll revisit the basics thread. I didn't see anything that was applicable. At this point I'm not concerned about the header. I just want to know i'm reading the data correctly by looking at sample of the data stream.
The thought that the data was fine coming in and then wasn't when sending to serial monitor also crossed my mind. So, I did have a program to look for the header. It was a series of nested while and if statements. It would never find the header indicating that the incoming data was not what I was expecting.

TheMemberFormerlyKnownAsAWOL:
if (Serial1.available() > 150) How big is the serial input buffer?

this was just one version where I was experimenting with this line. The if statement is usually looking for >0.

Thank you all for your input so far.

srnet:
Serila.read() just reads a byte of data, its carries out no 'conversion' on that byte, nor should it.

You can then print that received byte how you like, as a decimal number, hexadecimal, binary or indeed as an ASCII character.

This is what I thought going into the project... but no matter what I do, I don't seem to get the correct data.

Looks like you successfully read the header in the example you gave.
What were you expecting?
What did you actually get?

How’s the device connected?
Is it outputting RS232 or TTL logic levels? If RS232 you will need a converter.
What baud rate?
What parameters? Start/stop bits, parity?

jcaplins:
This is what I thought going into the project... but no matter what I do, I don't seem to get the correct data.

See example code in reply #6. If it doesn't pass the data from one port to another, you don't have a coding problem but a wiring problem.

On a side note; your use of a serial-to-usb converter raises alarm bells. Are you converting RS232 signals to USB? Or are you converting TTL signals to USB?

I'll try to answer as many question as I can here.
Output of device is rs232, 8N1, 3 byte header, 1 byte checksum

sterretje:
This should be the basics for testing; it writes the received byte as received without conversions.

void loop()

{
if (Serial1.available() > 0)
{
byte incomingByte = Serial1.read();
Serial2.write(incomingByte);
}
}



Not tested.

Using this code above, I have:
My device --> arduino Rx1 (pin 19) --> the code above --> Tx2 (pin 16) --> rs232 to usb converter --> a serial monitor program. the output looks like:

Using this code above, but change write to print, I still have:
My device --> arduino Rx1 (pin 19) --> the code above --> Tx2 (pin 16) --> rs232 to usb converter --> a serial monitor program. the output looks like:

mostly same code as above (serial.write), changing to Arduino serial monitor, so serial instead of serial2:
My device --> arduino Rx1 (pin 19) --> the code above --> usb cable to PC --> Arduino serial monitor. the output looks like:

mostly same code as above, but using (serial.print), reading with Arduino serial monitor, so still serial instead of serial2:
My device --> arduino Rx1 (pin 19) --> the code above --> usb cable to PC --> Arduino serial monitor. the output looks like:

same but with some spaces:

This is not using the Arduino at all. I can see where the header is and the data is correct.
My device --> rs232 to USb converter --> serial monitoring program:

Your device is a RS232 device; your Arduino is not. You will need a RS232-to-TTL converter between the device and the Arduino.

I don't see it mentioned anywhere in above reply.

And in the same line, the RS232-to-USB converter can't be used between an Arduino and the PC; you'll need a TTL-to-USB converter.

pcbbc:
Looks like you successfully read the header in the example you gave.
What were you expecting?
What did you actually get?

How’s the device connected?
Is it outputting RS232 or TTL logic levels? If RS232 you will need a converter.
What baud rate?
What parameters? Start/stop bits, parity?

I was not able to find the header using the arduino. I can only see the header by connecting the device directly to the converter, bypassing the Arduino altogether.

The device is outputting 9600 baud, 8N1.

sterretje:
Your device is a RS232 device; your Arduino is not. You will need a RS232-to-TTL converter between the device and the Arduino.

I don't see it mentioned anywhere in above reply.

Ahhh... I have not read that anywhere before, or completely missed it. That should solve my problem. I will look into a converter.

or...

Can I use the software.serial for RS232 input?

Thanks!

jcaplins:
Can I use the software.serial for RS232 input?

Software or hardware serial you will need some kind of translation hardware.
RS232 logic levels are:
0 +3v to +15v
1 -3v to -15v

So software serial wont help you with that. Also, if you’ve already connected them directly to your Arduino logic pins you may well have damaged it.

pcbbc:
So software serial wont help you with that.

It will not help with the levels but it does support inverse logic; see https://www.arduino.cc/en/Reference/SoftwareSerialConstructor.

@jcaplins

For the levels, one usually uses a MAX232. Alternatively, for the Arduino's RX pin, possibly a voltage divider and a diode; a little outside my league but I'm sure one can find schematics on the web.