Go Down

Topic: Serial transfers corrupted... (Read 3095 times) previous topic - next topic

Anders 2009

Hi,
this is probably a newbie question, but I would really appreciate any pointers to help me take the next step in solving a "mysterious" problem I have with serial transfer of a binary data file (190 bytes) from Mac -> Arduino.

I have tried to search this forum and the net, but I haven't found anything similar. That's why I think it's me and not the platform...

In short, I am trying to transfer a binary file, using 9600bps, from a Procesing Sketch to my Arduino. Once it didn't work immediately I started debugging and I have now discovered that even if I just echo what I receive in the Arduino back, the data gets "corrupted" all the time. Please see the log sample and code below.

To make it dead simple I just send one byte at a time to the Arduino and then echo them back as soon as I read them. Should be fine, right?

Serial transfers of sketches etc. work perfectly, so I am a bit at a loss. Any ideas, anyone?

Thanks/Anders

[font=Courier New][size=11]
____ DEBUG OUTPUT (from Processing sketch)

Wrote byte:      -1      hex: FF
Wrote byte:      -1      hex: FF
Wrote byte:      1      hex: 01
Wrote byte:      2      hex: 02
Wrote byte:      54      hex: 36
Wrote byte:      -1      hex: FF
Wrote byte:      -1      hex: FF
Wrote byte:      1      hex: 01
[glow]Wrote byte:      66      hex: 42
Wrote byte:      117      hex: 75
Wrote byte:      115      hex: 73
Wrote byte:      115      hex: 73[/glow]
...

Read byte val: -1      hex: FF
Read byte, val: -1      hex: FF
Read byte, val: 1      hex: 01
Read byte, val: 2      hex: 02
Read byte, val: 54      hex: 36
Read byte, val: -1      hex: FF
Read byte, val: -1      hex: FF
Read byte, val: 1      hex: 01
[glow]Read byte, val: 65      hex: 41 <<<<< WHAT? Should be value 66 (see above)
Read byte, val: 115      hex: 73 <<<<< CORRUPT FROM HERE ON
Read byte, val: 3      hex: 03
Read byte, val: 23      hex: 17[/glow]
...


____ CODE THAT SENDS (Processing)

 byte b[] = loadBytes("binary.file");
 for(int i = 0; i < b.length; i++) {
   myPort.write(b);
   println("Wrote byte:\t"+b+"\thex: "+hex(b));
 }



____ CODE THAT RECEIVES AND ECHOS BACK (Arduino)

// Read one byte and acknowledge by sending it back
//
byte ACKRead() {
 while(Serial.available() < 1) {
   // wait for data
 }
 byte b = Serial.read();
 Serial.write(b);
 return b;
}[/size][/font]



PaulS

Just out of curiosity, what happens if you add a delay(5); statement between the Serial.read() and the Serial.write()?

Mitch_CA

It's possible that you're dumping too much data onto the Arduino serial port before it has a chance to read it.
Try some handshaking; don't push all the data unless Arduino is ready to receive it.
That's where I would suspect.

Anders 2009

Wow!, thanks for the quick replies!

@Mitch_CA:
I have been suspecting some kind of buffer overrun as well. The Arduino ref. says 128bytes serial buffer and I know I am sending more. Unfortunately it does not say what happens when the buffer is full on the Arduino side. Will it block the sender from sending more until buffer is depleted?, or will it start corrupt/overwrite? I was guessing/hoping that it would block the sender from sending more data. Maybe it doesn't.

I will try to add some primitive handshaking/ACK to see if things get better.

@PaulS
I will add the delay just to see what happens.

Thanks guys!/Anders

Anders 2009

Ok,
I tried adding the delay(5)-statement without any effect. Saw exactly the same error.

Will try to whip up some primitive handshaking now to prevent buffer overflow.

Surely, this must be something that has been solved before. Anyone knows a good library to use (like "safe serial transfer")?

/Anders

Anders 2009

@AWOL: ARGH! Thanks man!
One bug down, although it didn't do the trick.
I changed the code to read as below, but the problem prevails. I will add the handshake as well and then maybe it will start working.

New version
Code: [Select]
// Read one byte and ACKknowledge by sending it back
byte ACKRead() {
 boolean dataPresent = false;
 while([glow]!dataPresent[/glow]) {
[glow]    if(Serial.available() >= 1)
     dataPresent = true;[/glow]
 }
 byte b = Serial.read();
 Serial.write(b);
 return b;
}


Thanks a bunch!/Anders

Anders 2009

hmmm... so maybe that wasn't a bug after all, but I'll keep the new code anyway. It is perhaps more verbose, but maybe easier to read.

Handshake next!/Anders


AWOL

Quote
Unfortunately it does not say what happens when the buffer is full on the Arduino side. Will it block the sender from sending more until buffer is depleted?, or will it start corrupt/overwrite? I was guessing/hoping that it would block the sender from sending more data. Maybe it doesn't.


It will simply overwrite - there's no flow-control built into the interface. It would be difficult to do anyway in a three wire interface, when you can't use XON/XOFF because you may be transmitting binary and not ASCII streams.

My first thought was a minor speed / bits-per-character mismatch, but that looks unlikely by the nature of the corruption.
My earlier suggestion was wrong anyway, sorry.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

Anders 2009

@AWOL. Thanks! That confirms my suspicions. I think some handshaking will solve it then.

I won't have time to work on it anymore before next week, but thanks for all the help!

/Anders

Mitch_CA

Maybe dump something like this into your send (processing) code...

Code: [Select]

byte b[] = loadBytes("binary.file");
for(int i = 0; i < b.length; i++) {
  myPort.write(b[i]);
  println("Wrote byte:\t"+b[i]+"\thex: "+hex(b[i]));
  while(myPort.available() < 1) {
    // wait for echo data
  }
  byte Rx = myPort.read();
  if (Rx != b) println("Error: Received " + Rx);
}


Anders 2009

@Mitch_CA: Thanks!, That's a good idea.

I was starting down a more difficult route, trying to orchestrate the send routine and the serialEvent(...) callbacks you get in Processing. Your suggestion is much more straight forward. I'll try that first (next week).

Thanks again!/Anders

Grumpy_Mike

#11
Jan 04, 2010, 03:22 pm Last Edit: Jan 05, 2010, 03:21 pm by Grumpy_Mike Reason: 1
Sorry (see below)
Comments removed.

Mitch_CA

Check details Mike... he's just displaying the data in two formats.  There is a discrepancy between the "read" data and "write" data that doesn't matchup.

Go Up