next topic »

[on:]

Hi,
this is probably a newbie question, but I would really appreciate any pointers to help me take the next step in solving a "mysterious" problem I have with serial transfer of a binary data file (190 bytes) from Mac -> Arduino.

I have tried to search this forum and the net, but I haven't found anything similar. That's why I think it's me and not the platform...

In short, I am trying to transfer a binary file, using 9600bps, from a Procesing Sketch to my Arduino. Once it didn't work immediately I started debugging and I have now discovered that even if I just echo what I receive in the Arduino back, the data gets "corrupted" all the time. Please see the log sample and code below.

To make it dead simple I just send one byte at a time to the Arduino and then echo them back as soon as I read them. Should be fine, right?

Serial transfers of sketches etc. work perfectly, so I am a bit at a loss. Any ideas, anyone?

Thanks/Anders

[size=11]
____ DEBUG OUTPUT (from Processing sketch)

Wrote byte:      -1      hex: FF
Wrote byte:      -1      hex: FF
Wrote byte:      1      hex: 01
Wrote byte:      2      hex: 02
Wrote byte:      54      hex: 36
Wrote byte:      -1      hex: FF
Wrote byte:      -1      hex: FF
Wrote byte:      1      hex: 01
[glow]Wrote byte:      66      hex: 42
Wrote byte:      117      hex: 75
Wrote byte:      115      hex: 73
Wrote byte:      115      hex: 73[/glow]
...

Read byte val: -1      hex: FF
Read byte, val: -1      hex: FF
Read byte, val: 1      hex: 01
Read byte, val: 2      hex: 02
Read byte, val: 54      hex: 36
Read byte, val: -1      hex: FF
Read byte, val: -1      hex: FF
Read byte, val: 1      hex: 01
[glow]Read byte, val: 65      hex: 41 <<<<< WHAT? Should be value 66 (see above)
Read byte, val: 115      hex: 73 <<<<< CORRUPT FROM HERE ON
Read byte, val: 3      hex: 03
Read byte, val: 23      hex: 17[/glow]
...


____ CODE THAT SENDS (Processing)

  byte b[] = loadBytes("binary.file");
  for(int i = 0; i < b.length; i++) {
    myPort.write(b);
    println("Wrote byte:\t"+b+"\thex: "+hex(b));
  }



____ CODE THAT RECEIVES AND ECHOS BACK (Arduino)

// Read one byte and acknowledge by sending it back
//
byte ACKRead() {
  while(Serial.available() < 1) {
    // wait for data
  }
  byte b = Serial.read();
  Serial.write(b);
  return b;
}[/size]

[Reply #1 on:]

Just out of curiosity, what happens if you add a delay(5); statement between the Serial.read() and the Serial.write()?

[Reply #2 on:]

It's possible that you're dumping too much data onto the Arduino serial port before it has a chance to read it.
Try some handshaking; don't push all the data unless Arduino is ready to receive it.
That's where I would suspect.

[Reply #3 on:]

Wow!, thanks for the quick replies!

@Mitch_CA:
I have been suspecting some kind of buffer overrun as well. The Arduino ref. says 128bytes serial buffer and I know I am sending more. Unfortunately it does not say what happens when the buffer is full on the Arduino side. Will it block the sender from sending more until buffer is depleted?, or will it start corrupt/overwrite? I was guessing/hoping that it would block the sender from sending more data. Maybe it doesn't.

I will try to add some primitive handshaking/ACK to see if things get better.

@PaulS
I will add the delay just to see what happens.

Thanks guys!/Anders

[Reply #4 on:]

Ok,
I tried adding the delay(5)-statement without any effect. Saw exactly the same error.

Will try to whip up some primitive handshaking now to prevent buffer overflow.

Surely, this must be something that has been solved before. Anyone knows a good library to use (like "safe serial transfer")?

/Anders

[Reply #5 on:]

@AWOL: ARGH! Thanks man!
One bug down, although it didn't do the trick.
I changed the code to read as below, but the problem prevails. I will add the handshake as well and then maybe it will start working.

New version

Code:

// Read one byte and ACKknowledge by sending it back
byte ACKRead() {
  boolean dataPresent = false;
  while([glow]!dataPresent[/glow]) {
[glow]    if(Serial.available() >= 1)
      dataPresent = true;[/glow]
  }
  byte b = Serial.read();
  Serial.write(b);
  return b;
}

Thanks a bunch!/Anders

[Reply #6 on:]

hmmm... so maybe that wasn't a bug after all, but I'll keep the new code anyway. It is perhaps more verbose, but maybe easier to read.

Handshake next!/Anders

[Reply #7 on:]

Unfortunately it does not say what happens when the buffer is full on the Arduino side. Will it block the sender from sending more until buffer is depleted?, or will it start corrupt/overwrite? I was guessing/hoping that it would block the sender from sending more data. Maybe it doesn't.

It will simply overwrite - there's no flow-control built into the interface. It would be difficult to do anyway in a three wire interface, when you can't use XON/XOFF because you may be transmitting binary and not ASCII streams.

My first thought was a minor speed / bits-per-character mismatch, but that looks unlikely by the nature of the corruption.
My earlier suggestion was wrong anyway, sorry.

[Reply #8 on:]

@AWOL. Thanks! That confirms my suspicions. I think some handshaking will solve it then.

I won't have time to work on it anymore before next week, but thanks for all the help!

/Anders

[Reply #9 on:]

Maybe dump something like this into your send (processing) code...

Code:

byte b[] = loadBytes("binary.file");
 for(int i = 0; i < b.length; i++) {
   myPort.write(b[i]);
   println("Wrote byte:\t"+b[i]+"\thex: "+hex(b[i]));
   while(myPort.available() < 1) {
     // wait for echo data
   }
   byte Rx = myPort.read();
   if (Rx != b) println("Error: Received " + Rx);
 }

[Reply #10 on:]

@Mitch_CA: Thanks!, That's a good idea.

I was starting down a more difficult route, trying to orchestrate the send routine and the serialEvent(...) callbacks you get in Processing. Your suggestion is much more straight forward. I'll try that first (next week).

Thanks again!/Anders

[Reply #11 on:]

Sorry (see below)
Comments removed.

[Reply #12 on:]

Check details Mike... he's just displaying the data in two formats.  There is a discrepancy between the "read" data and "write" data that doesn't matchup.