How to ignore "00" HexChar on Arduino Serial

I'm having problems with data received through the serial, and I believe this is happening because "00" HEX bytes is received by the arduino through the serial port.

This is the Simplest code:

#include <SoftwareSerial.h>
SoftwareSerial mySerial(10, 9); // RX, TX


void setup()  
{    Serial.begin(9600);
     mySerial.begin (9600);

}

void loop() // run over and over
{ 
             
     if (mySerial.available() > 0 ) {;
     lcd.setCursor (0,0) ;
     lcd.clear();
     Serial.println(mySerial.readString());
}


    
}

Data is sent from an MP3 module with song number and name information

The real problem is that the arduino is not working correctly with the data received in the part of the data that contains the song name.
It displays some bugs such as showing the same name for another 6 or 8 songs ahead or not displaying the name.
I discovered the possible problem when I noticed that there was a space between each character. I tried to copy the name of the song to notepad and it was possible if copy letter by letter. when I tried to copy along with this strange space between the letters it was not possible.

d9fe3e8106550d469799daee1da135aaa246a917_2_690x386

After self burn my brain investigating I decide to open the serial monitor and connect the MP3 module directly to the computer where it works correctly. I used the Stock module configuration software to read the information.

This is how the info works when connect directy on computer by TTL converter (UART TO USB)

The Question is that the Factory software on Computer Ignores the "00".

I tried using the arduino serial monitor to check if the zero was ignored there too and the result was that weird space between the letters that this is a character that should be ignored

Connected By UART to USB Directly.

How can i do to Serial dont read this "00" ?

you don't know that.

don't use readString() and just read what's coming byte by byte. You'll know exactly what's the flow
➜ run this with the Arduino Serial Monitor opened at 115200 bauds

#include <SoftwareSerial.h>
SoftwareSerial mySerial(10, 9); // RX, TX
void setup()
{
  Serial.begin(115200);
  mySerial.begin (9600);
}

void loop() // run over and over
{
  static byte nbCol = 0;
  if (mySerial.available() > 0 ) {
    int r = Serial.read();
    if (r < 0x10) Serial.write('0');
    Serial.print(mySerial.read(), HEX);
    Serial.write(' ');
    nbCol = (nbCol + 1) % 10;
    if (nbCol == 0) Serial.println();
  }
}
  • how do you send commands to the module?
  • can you clarify the exact wiring of your system?
1 Like

I really do not know. But this bug is just in the song name, and that byte "00" creates an invisible character. That's why I'm investigating. The data normally comes from the computer.

I Will do this now.

I was using variables String , but now I'm using text converted to HEX.
For example:

Serial.write ("AT+CE\r\n",HEX) \\ This is to Next Track song

This is not a correct use of write and your module is not on Serial

Can you post the full test sketch sending the commands?

1 Like

I think that the song names are encoded in some Unicode format like UTF-8 or UTF-16. These use a variable number of bytes per character instead of the usual 8 bits so that characters from other languages and alphabets can be represented such as Cyrillic, Kanji, etc. I'm no expert but I would guess that if the first byte is zero, the second byte is standard ASCII.

1 Like

Yes .. worth checking the MP3 files with a hex editor or a tag editor to look for ( any) differences that you can then accommodate in your sketch

1 Like

UTF8 would be variable byte length so typical ASCII would be on 1 byte — but UTF16 or UCS-2 would indeed lead to 2 bytes minimum per char. The endianness can also play tricks in that case.

1 Like

You screwed yourself with this;

 if (mySerial.available() > 0 ) {;

That automatic ; at the end of the line.... no, the { is to keep it open, you ended it.

{
so the lines inside play and end with ;
and one branch can have many lines before it ends ;
}

what you did, those lcd lines and the print happen no matter what is read.

Just another reason to open and close braces on the same indent level, it's easy to keep clear what is inside where and matching braces.

Nick Gammon's Serial tutorial, especially the state machine example would be like a new set of off-road tires for you and anything serial.
http://www.gammon.com.au/serial

State machines run code to match conditions from on/off to is the gate open, is the fan on, has 30 seconds passed yet, and all of those checks "live". When hacking text or data I put state machines inside of state machines to read one file and write an edited version. They can be simple to detailed as proccesses can have modes/steps.

1 Like

The ; should not be there, but I don't think it's doing any harm. It is inside the { }, so will not end the if-statement prematurely. It's just an empty statement and will be ignored. If the ; had been before the {, then you would be correct, it would have prematurely ended the if-statement.

1 Like

Correct it’s just an empty useless statement within the {} compound statement that the if is executing when the condition is true

1 Like

Can you share a link to documentation about the mp3 module and the data formats it sends?

There is a danger here that it might be sending the song title as raw data without any meta-data to tell the Arduino what encoding had been used. This will make it very hard for your code to correctly interpret and display the data, because different MP3 files might use different encodings.

1 Like

Yes, This is! I was reading the service manual and this talk about UTF-8 to chinese chars.

I'm checking all files to do this, but the most promising result is how the serial monitor shows me the bytes with this "00" byte.

I still don't know much about this, i need to study more.

I'll watch this.
Im Basing on Youtube videos to Learn about how use the serial port.

Everything I try to do is hard for me cuz I'm using SoftwareSerial and it's so much bad, don't work like native serial.

I'm using it because I think the same as you. I watch the videos on youtube to learn how to use the functions, but I understand the logic and if I see that I can do it better I do

I managed to solve the case. From what I understand in fact the byte "00" ended up corrupting the memory of the arduino, causing it to create a false memory based on that character.
To ignore the "00" I chose to use the following sketch

#include <Wire.h>
#include <SoftwareSerial.h>
SoftwareSerial mySerial(10, 9); // RX, TX


String serialinput;

void setup()  
{
     Serial.begin (115200);
     mySerial.begin (9600);
}


void loop(){  

   while (mySerial.available() > 0 ) {
   serialinput.concat(mySerial.readStringUntil(00));}    /// READ THE STRING UNTIL MATCH THE FIRST 00 byte, and this byte is removed of buffer
   Serial.print (serialinput);
   serialinput = "";  /// Clean the variable to wait a new info read 


}

This way while there is data being received on the serial port it gets stuck in the "while" loop.

The "readStringUntil" function discards the terminator character from the buffer

so it will loop ignoring all 00 until "While" is "False"

while (mySerial.available() > 0 )

When it get out the loop, all received data was writted in the string "serialinput" by the "Concat" function

This Part

Serial.print (serialinput);
   serialinput = "";

Writes all data to the serial monitor and clears after that so that the Concat function does not continue Concatenating all received data.

This is the solution. Working Normally now!

This is the data received by arduino without the "00" HEX char.

That’s a poor solution but if you are happy with it, go for it!

Only problem I see is if you get messages with >64 characters since reading is done at 12x the write speed, 115200 vs 9600.

At 115200 a single char takes 86.8usec to complete where at 9600 it takes 1041.7usec to complete.

Once the input buffer fills, your new reads will miss chars.

OTOH if you never get a long-enough -to-choke message, you're good!

1 Like

For me it's working normally. Can lose some bytes received but isn't happening; I think that is cuz is in Loop.

If I need to apply it in some program to run this specific code at specific times I will use the Whille function based on if and millis for dont loss the data.

I know it's not the best solution, but it's the simplest for me

No prob, it reads slow, prints fast.