printing raw data

Hi,

I am using the following code to print an array of 4 values to the serial port as raw bytes.

  unsigned int valOut;  // print out the results from a strike
  unsigned char cOut;
  for (char i=0; i<MAXPADS; i++) {
    valOut=(unsigned int)piezoSum[i];
    // now put the output values into the lower 7 bits of 2 bytes, high part first.
    cOut=((unsigned char)(valOut >> 7)) & 0x7f;
    Serial.write(cOut);
    //Serial.print(" ");
    cOut=(unsigned char)valOut & 0x7f;
    if (i==3) {
      cOut|=0x80; // last character has the high bit set as a flag.
    }
    Serial.write(cOut);
    //Serial.print(" ");
  }

The idea is that each of the (integer) values in piezoSum[] is converted to two 7 bit bytes, with the top bit of the last byte in the sequence set as a marker. The problem I'm having is that sometimes the arduino program seems to output less bytes than it should - when the marked byte is detected by the program reading the port (in linux), there are less bytes in the buffer than there should be.

The problem is almost certainly with this bit of code - when I put it into debug mode which just prints the numbers as strings, it works fine. I've also tried just catting the port through the hexdump utility to see if there's a fault with the program that receives the bytes, and there is still the same problem.

Thanks for any help with this,

andy

(modified once to improve formatting)

Is this supposed to be an array access...

valOut=(unsigned int)piezoSum [ i ] ;

Yes - piezoSum is an a array of long integers which is set by another bit of the program.

That snippet prints two bytes every time through the loop. If you go through the loop four times, it prints a total of eight bytes. All of the bytes will have '0' as the most significant bit except for the last byte, which will have '1' as the MSB.

Make a test program that uses Serial.print and shows hex values to see if it is doing what you think it should.

#define MAXPADS 4
unsigned int piezoSum[MAXPADS] = {0x1234, 0x5678, 0x9abc, 0xdef0};

void setup()
{
    Serial.begin(9600);
    unsigned int valOut;  // print out the results from a strike
    unsigned char cOut;
    for (int i = 0; i < MAXPADS; i++) {
        
        valOut = piezoSum[i];
        Serial.print("valOut = 0x");
        Serial.print(valOut,HEX);Serial.print(" ==> 0x");

        cOut = (valOut >> 7) & 0x7f;
        Serial.print(cOut, HEX);
        Serial.print(" 0x");
        cOut = valOut & 0x7f;
        if (i == MAXPADS-1) {
            cOut|=0x80; // last character has the high bit set as a flag.
        }
        Serial.println(cOut, HEX);
    }
}
void loop(){}

Output:


[color=blue]valOut = 0x1234 ==> 0x24 0x34
valOut = 0x5678 ==> 0x2C 0x78
valOut = 0x9ABC ==> 0x35 0x3C
valOut = 0xDEF0 ==> 0x3D 0xF0[/color]

When you are sure that it gives the correct number of bytes (and the bytes have the correct values) then go back and eliminate the extra print stuff and use Serial.write(cOut) (or Serial.print(cOut,BYTE) or whatever) to put the binary values out to the port.

Regards,

Dave

Footnote:
I know that sometimes you have to use casts in C++ to keep the compiler happy, but when I see a bunch of superfluous casts sprinkled throughout the landscape, it makes me fraught. I'm wondering why you thought you needed them? Did the compiler give some kind of annoying messages that you wanted to suppress? Or what?

davekw7x:
That snippet prints two bytes every time through the loop. If you go through the loop four times, it prints a total of eight bytes. All of the bytes will have '0' as the most significant bit except for the last byte, which will have '1' as the MSB.

Make a test program that uses Serial.print and shows hex values to see if it is doing what you think it should.

I'll give this a try - thanks.

davekw7x:
Footnote:
I know that sometimes you have to use casts in C++ to keep the compiler happy, but when I see a bunch of superfluous casts sprinkled throughout the landscape, it makes me fraught. I'm wondering why you thought you needed them? Did the compiler give some kind of annoying messages that you wanted to suppress? Or what?

The only reason I put them in was because I wasn't sure how the C++ compiler deals with type conversion between signed and unsigned, and short and long, variables. I wondered if there might be some kind of overflow error going on. They don't seem to make any difference, so maybe I should take them out.

I've written some test code, and there is still a problem. This is the code I wrote:

#define MAXPADS 4
#define MINVAL 0
#define MAXVAL 127
#define MAXDEL 250
#define MINDEL 250
unsigned long piezoSum[MAXPADS] = {0x1234, 0x5678, 0x9abc, 0xdef0};

void setup() {
  Serial.begin(19200);
}

void loop() {
  unsigned int valOut;  // print out the results from a strike
  unsigned char cOut;
  for (char i=0; i<MAXPADS; i++) {
    piezoSum[i] = random(MINVAL,MAXVAL);
  }
  for (char i=0; i<MAXPADS; i++) {
    valOut=piezoSum[i];
    // now put the output values into the lower 7 bits of 2 bytes, high part first.
    cOut=(valOut >> 7) & 0x7f;
    Serial.write(cOut);
    //Serial.print(" ");
    cOut=valOut & 0x7f;
    if (i==3) {
      cOut|=0x80; // last character has the high bit set as a flag.
    }
    Serial.write(cOut);
    //Serial.print(" ");
  }
  delay(random(MINDEL,MAXDEL));
}

I've tried various values for MAXVAL, MINVAL, MAXDEL, and MINDEL. The timing doesn't seem to make much difference, but the numbers being sent do make a difference - if MINVAL and MAXVAL are both zero then it runs fine, but with MINVAL=0 and MAXVAL=127, the number of bytes received before a mark byte (i.e. one with the high bit set) changes unpredictably. Below is a hexdump of the output with MINVAL=0, MAXVAL=127, and MINDEL=MAXDEL=250:

00000150  00 a6 00 5a 00 18 00 00  00 88 00 62 00 7b 00 19  |...Z.......b.{..|
00000160  00 87 00 23 00 23 00 6e  00 b5 00 32 00 11 00 1f  |...#.#.n...2....|
00000170  00 a5 00 43 00 0b 00 60  00 a4 00 24 00 3b 00 40  |...C...`...$.;.@|
00000180  00 c8 00 5e 00 08 00 33  00 88 00 2d 00 2f 00 01  |...^...3...-./..|
00000190  00 ed 00 0b 00 74 00 17  00 da 00 02 00 64 00 77  |.....t.......d.w|
000001a0  00 94 00 22 00 6f 00 35  00 be 00 23 00 23 00 43  |...".o.5...#.#.C|
000001b0  00 cf 00 41 00 5f 00 23  00 a7 00 78 00 4c 00 61  |...A._.#...x.L.a|
000001c0  00 a5 00 60 00 11 00 7e  00 81 00 34 00 33 00 69  |...`...~...4.3.i|
000001d0  00 b9 00 40 00 7e 00 7c  00 b3 00 4b 00 56 00 74  |...@.~.|...K.V.t|
000001e0  00 97 00 7b 00 49 00 49  00 db 00 07 00 1e 00 66  |...{.I.I.......f|
000001f0  00 ac 00 68 00 4b 00 22  00 96 00 0a 00 59 00 54  |...h.K.".....Y.T|
00000200  00 e5 00 2f 00 6f 00 00  8c 00 22 00 61 00 04 00  |.../.o....".a...|
00000210  80 00 77 00 0e 00 42 00  f9 00 69 00 2a 00 23 00  |..w...B...i.*.#.|

You can see from this that it is in a stable pattern with the mark byte second from the left of each block, and then shifts at around 0200 to another pattern with the mark byte at the left of the block.

I still can't work out why this isn't working - it's just meant to split the lower 14 bits of each value in piezoSum across two bytes, and send them out with a marker byte at the end of each block so the program interpreting it knows where it is in the sequence.

Cheers,

andy

I've cracked the problem, and it seems to be a bug in the arduino Serial library. For some reason, serial.write() seems to fail on the number 3.

This code works:

void setup() {
  Serial.begin(19200);
}

void loop() {
  unsigned int valOut;  // print out the results from a strike
  unsigned char cOut;
  cOut=0x04;
  Serial.write(cOut);
  cOut=0x83;
  Serial.write(cOut);
  delay(random(MINDEL,MAXDEL));
}

But this doesn't:

void setup() {
  Serial.begin(19200);
}

void loop() {
  unsigned int valOut;  // print out the results from a strike
  unsigned char cOut;
  cOut=0x03;
  Serial.write(cOut);
  cOut=0x83;
  Serial.write(cOut);
  delay(random(MINDEL,MAXDEL));
}

It just prints a series of 83 characters with no 03's at all:

00000000  83 83 83 83 83 83 83 83  83 83 83 83 83 83 83 83  |................|
00000010  83 83 83 83 83 83 83 83  83 83 83 83 83 83 83 83  |................|
00000020  83 83 83 83 83 83 83 83  83 83 83 83 83 83 83 83  |................|
00000030  83 83 83 83 83 83 83 83  83 83 83 83 83 83 83 83  |................|

I'll investigate a bit more and then maybe post a bug report. I'm using arduino version 0022.

highfellow:
I've cracked the problem, and it seems to be a bug in the arduino Serial library. For some reason, serial.write() seems to fail on the number 3.

How are you reading the bytes on your PC? Is the port set up in "raw" mode? (0x03 is Ctrl-c and maybe your PC port does something special with Ctrl-C)

Regards,

Dave

That's quite possible. I started a bug report here:

http://arduino.cc/forum/index.php/topic,59532.0.html

So any further replies to this should go there probably.

I tested the program with the usb serial set to raw mode (stty -F /dev/ttyUSB0 19200 raw), and it's working fine, so it was the ctrl-C thing that was causing the problem. Thanks to davekw7x for pointing this out!