string broken when Serial.read() is copied to char array

Hi guys, I have a problem of corrupted string when I copy the char into an char array, let see it....

Im processing on every loop call a Bluetooth serial (HM-10) with this code:

while (BTSerial.available() > 0) {
  char c = BTSerial.read();
  Serial.print(c);  
}

And the string looks OK

OK+DISISOK+DISC:00000000:00000000000000000000000000000000:0000000000:
F8042EDEAC33:**-089OK+DISC:00000000:00000000000000000000000000000000:0000000000:6A8A3D7E94C7:-078**OK+DISCE

But really what I want is add it to a char array to process it later, then my code should look like this:

char BTBuffer[300]; // I tried more length and its NOT about overflow.
int BTBufferIndex;


BTBufferIndex = 0;
memset(BTBuffer, 0, sizeof(BTBuffer));


// eventually in the code I do...

while (BTSerial.available() > 0) {
  char c = BTSerial.read();
  BTBuffer[BTBufferIndex] = c;
  BTBufferIndex++;
}

Eventually in the code I check the value of BTBuffer and the output is:
(usually do similar outputs around the same area (rssi) but almost all of them look similar)

OK+DISISOK+DISC:00000000:00000000000000000000000000000**00DO00OK+DISCE**

Other broken strings are:

OK+DISISOK+DISC:00000000:00000000000000000000000000000**:OK00OK+DISCE**

It never finish, tometimes I get the RSSI value AFTER the OK+DISCE, something like

...00000000**:-0OK+DISCE89**....

I tried few days but I cannot understand where is making some kind of delay writing or loosing the sync.
Any help?

Thanks!

try memset(BTBuffer, '/0', 300);

To print a string with functions like Serial.print() you need a NULL character on the end of the array. After you've recognised the input (seen a carriage-return for example) then add a '\0' character at the BufferIndex position.

mistergreen:
try memset(BTBuffer, '/0', 300);

...by all means, but use a backslash

Easiest way to add a null is to write one after each character that you've just read:

 char c = BTSerial.read();
  BTBuffer[BTBufferIndex] = c;
  BTBufferIndex++;
  BTBuffer[BTBufferIndex] = 0;

Pete

The safest way to do this is:

while (BTSerial.available() > 0) {
  char c = BTSerial.read();
  if (BTBufferIndex < 300-1)
  {
    BTBuffer[BTBufferIndex++] = c;
    BTBuffer[BTBufferIndex] = 0 ;
  }
}

ie always write the null after putting a character in, but also always check before doing so if there
is enough room.

Learn the ++/-- postfix and prefix operators, they are normally used for buffer operations like this.

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data.

...R

Hi guys, thanks you all for the answers, really appreciate the great arduino community.

I was trying your improvements but is happening exactly the same. Im gonna extend the code to give a better understanding.

void setBTState(BTState nextState) {
  BTBufferIndex = 0;
  memset(BTBuffer, '\0', 300);
  currentBTState = nextState;  
}

void pumpBTBuffer() {
  char c = BTSerial.read();
  if (BTBufferIndex < 300 -1) {
    BTBuffer[BTBufferIndex++] = c;
    BTBuffer[BTBufferIndex] = 0 ;
  }  
  Serial.println(BTBuffer);
}

void configBT() {
  btCommand("AT+NAMEClueless");
  btCommand("AT+ROLE1");
  btCommand("AT+IMME1");
  btCommand("AT+RESET");
}

void btCommand(const char* at) {
  BTSerial.write(at);
  delay(100);
}




void setup() {
  Serial.begin(9600);
  BTSerial.begin(9600);
  configBT();
  delay(1000);
  setBTState(INQ_START);
}

void loop() {

  switch(currentBTState) {
    case INQ_START:
      if (!isBTScanning) {
        isBTScanning = true;
        btCommand("AT+DISI?");
      }
        
      while (BTSerial.available() > 0) {
        pumpBTBuffer();
      }

      break;      
  }  

}

This is spitting the same problems

OK+Set:CluelessOK+Set:1OK+Set:1OK+RESETOK+DISISOK+DISC:00000000:00000000000000000000000000000:OK+00OK+DISCE

OK+Set:CluelessOK+Set:1OK+Set:1OK+RESETOK+DISISOK+DISC:00000000:000000000000000000000000000000AOK+DI0COK+DISCE

The expected (like printing the serial read step to step) should be something like....

OK+DISISOK+DISC:00000000:00000000000000000000000000000000:0000000000:
F8042EDEAC33:-089OK+DISCE

Robin2:
Have a look at the examples in Serial Input Basics - simple reliable ways to receive data.

...R

I saw that post few days ago and next to a reference in the book Arduino cookbook and i think im following the same style.

It look like (Always on that part)... like a synchronisation problem saving in the array and like only few (last) calls are done, of course and probably is not that but look like... is mising too much character (UUID) and start mixing it with the OK+DISCE that never fail, only sometimes split it.

In setup(), you send a number of commands to the BT module. That module will reply to all of them and that is what you will collect in loop().

Send one command, read the reply. Send next command, read the reply. And so on. After each reply, clear the buffer. You will need to find a way to determine when the reply is complete (e.g. it might end with carriage return and/or linefeed).

You also print the received text every time that you receive a single character. So your displayed text will more than likely look like

O
OK
OK+
OK+S
OK+Se
OK+Set
...
...

jjosealonso:
I saw that post few days ago and next to a reference in the book Arduino cookbook and i think im following the same style.

It look like (Always on that part)... like a synchronisation problem saving in the array

Post the code that you tried using the example from my link.

...R

sterretje:
In setup(), you send a number of commands to the BT module. That module will reply to all of them and that is what you will collect in loop().

Send one command, read the reply. Send next command, read the reply. And so on. After each reply, clear the buffer. You will need to find a way to determine when the reply is complete (e.g. it might end with carriage return and/or linefeed).

You also print the received text every time that you receive a single character. So your displayed text will more than likely look like

O

OK
OK+
OK+S
OK+Se
OK+Set
...
...

Yes but I pasted the last one.

But the commands in order is not the problem because if I print the character read everything come in order. Its in the copy process where it is copied wrongly

If I do Serial.print(BTSerial.read()) inside of a while Serial.available() > 0, and inside of my loop func it print perfect the string in the proper format then IM SURE that the BT module is returning the data in the proper order.

About command by command:
I want to analyse the buffer after the first OK+DISIS, im cleaning the buffer after that and changing the state, I have a statemachine (all comented out now) for the parsing.

BUT is no reason to justify why is copying wrongly.

Hi guys again.

I just discover the reason why the copy was corrupted and its quite weird for me, maybe you guys with a deep experience in Android can know why.

If I keep doing this portion of code:

void pumpBTBuffer() {
  char c = BTSerial.read();
  if (BTBufferIndex < 300 -1) {
    BTBuffer[BTBufferIndex++] = c;
    BTBuffer[BTBufferIndex] = 0 ;
  }  
 Serial.println(BTBuffer);
}

For some reason after few dozens of serial.print the char copy start failing and look like info in the middle is missing, also look like the last queued serial.print are processed right, thats why always close with OK+DISCE or at least with a proper DISCE.

Example:
OK+DISISOK+DISC:00000000:00000000000000000000000000000:OK00OK+DISCE

Instead of:
OK+DISISOK+DISC:00000000:00000000000000000000000000000000:0000000000:
F8042EDEAC33:-089OK+DISCE

I discovered that IF i delete the serial.prints and I do a print each 6000ms, the string was copied properly.

Then, in serial.print, when you overflow with a lot of them probably is causing a bug or a recognised bug in the system. I heard that serial.print (transmit buffer) is async, and maybe thats why.

Somebody knew about this?

a Serial.print(BTBuffer) on each loop of the while, for some reason the system is so busy sending through the serial

jjosealonso:
For some reason after few dozens of serial.print the char copy start failing and look like info in the middle is missing,

That suggests that counting is not a robust way to detect the end of a message.

Did you try the examples in the link I gave you?

...R

Robin2:
That suggests that counting is not a robust way to detect the end of a message.

Did you try the examples in the link I gave you?

...R

I had a read when you told me but im seeing that im doing the same technic, isnt ?
im missing something?

Anyway, the issue is focus in the copy char section, I didnt test properly if is a problem printing or a real problem in the char array since my prev example never stopped printing, maybe after leaving a pause the line was fully.

But I believe by some changes that I did, that the problem is in the copying. whats your opinion of whats is happening technically?

jjosealonso:
im missing something?

The thing that is missing for me is that you have not posted the version of your program in which you used one of my examples. If you post that then I can help with it.

...R

I will have another review this evenning.
Thanks.

sterretje's reply #9 identifies the problematic code:

You also print the received text every time that you receive a single character.

This means that you are printing 1+2+3+ ... +n characters. There is a formula for that sum: (n2 + n) / 2.

For a response of 144 characters, you are trying to print more than 10,000 characters. -_- Don't do that. Here's what's happening:

After the 1st character comes in, you print 1 character. That character (and a newline) is copied to the Serial output buffer, and the print returns. The 1st characters starts to go out, but the output buffer still has 1 character (a newline).

After the 2nd character comes in, you print 2 characters. The 1st character has gone out, but the newline has not, so the 1st and 2nd characters (and a newline) are copied to the output buffer, and the print returns. The output buffer contains 3 characters.

After the 3rd character comes in, you print 3 characters. The newline character from the 1st print has gone out, but the output buffer still contains the 3 characters from the 2nd print. This 3rd print adds 4 characters to the output buffer and returns. The output buffer contains 7 characters.

After the 4th character comes in, you print 4 characters. The 1st character from the 2nd print has gone out, but the output buffer still contains the 6 characters. This 4th print adds 5 characters to the output buffer and returns. The output buffer contains 11 characters.

...

After the 11th character comes in, the 64-character output buffer is full. You will have to wait 12 character times (12ms) to print the 12 characters you have so far (11 characters of the response plus a newline). During that time, 12 new characters will arrive.

Then you will read the 12th character and print the 12 characters in BTbuffer (plus a newline). But the output buffer is still full, so that Serial.print will have to wait 13 character times before it can return. Aaaaand another 13 characters will arrive. There are now 25 characters in the input buffer.

Then you will read the 13th character and print 14. But the output buffer is still full, so that Serial.print will have to wait 14 character times before it can return. Aaaaand another 14 characters arrive. There are now 39 characters in the input buffer.

Ditto 14th... There are now 53 characters in the input buffer.

Ditto 15th... But this is where it fails, because the input buffer only has room for 64 characters. You have lost characters 65..68.

You are simply printing too much. It's not a problem copying characters. The characters were dropped, because the input buffer overflowed while the output buffer was being emptied.

Instead, only print the response when the entire line has arrived:

void pumpBTBuffer() {
  char c = BTSerial.read();
  if (BTBufferIndex < sizeof(BTBuffer) -1)
    BTBuffer[BTBufferIndex++] = c;

  if (c == '\n') {
    BTBuffer[BTBufferIndex] = '\0'; // NUL-terminate
    Serial.println(BTBuffer);

    // This is a good time to return 'true' or change state
  }
}

Using sizeof allows you to change the buffer size and not have to search and replace all the 300s (or 299s) with the new size.

Robin2's code does not have this problem.

Cheers,
/dev

P.S. This may be off by a character or two... :slight_smile:

Sounds like Schlemiel the Painter's algorithm.

MorganS:
Schlemiel the Painter's algorithm.

Sounds great - do you have a link for it ?

...R