Serial read more than 64 bytes?

I'm still working on my ticker that I created with some nice LED displays. With Paul's help, I was able to scroll serial data across my display. Help implement queue for ticker? - Programming Questions - Arduino Forum

Now I find I can't read more than 64 bytes at one time.

Here is the way I am reading the data:

void loop(){
  if (Serial.available() > 0 ){
       for (byte i = 0; i < 8; i++){
         message[i] = message[i + 1];
       }
       char letter = Serial.read();
       message[7] = letter;
       showMessage();
  }
}

this scrolls fine until I get to the 64th byte and then it stops. I'm obviously having trouble understanding how serial works, and I guess I can't read the serial data fast enough and print to the display at the same time. Can anyone show me some examples on how to catch the data in an array quickly and THEN send it out to the display.

Thanks,

Jimmy

It's hard to say without seeing all your code.

http://snippets-r-us.com/

For example, how big is message? What does showMessage do?

I hop you don't have delay() in your code. You can't sit around delaying, and simultaneously expect to receive a lot of serial data.

I note that the serial buffer size is 64 bytes (if you have 1000 bytes or more of RAM) so this suggests you aren't pulling the message out of the holding buffer fast enough. Which is very likely if you are using delay.

How to use this forum

@ Nick G... +1

Doc

Hi nick,

Thanks for the comments. I get all the jokes about the snippets and everything, but I can't fit all the code in here, and most of it is all wire.transmit this and wire.end transmission that. If I put all the code in no one would read the whole thing and help me. It's more fun to be sarcastic.

I'm sitting here looking at your example from your website, and I get the same problem with your code. Gammon Forum : Electronics : Microprocessors : How to process incoming serial data without blocking

showMessage looks like this:

void showMessage(){
  for(int i = 8; i > 0; i--){
    displayChar(message[i-1], i-1, 0);
  }
}

where displayChar is a bunch of i2C stuff that allows me to say 'write a character in a position (0-7),
of one of a number of led displays)

like so

void displayChar(char myChar, int myPos, byte myDisp){
     …. a bunch of wire. calls that write a char in a position…..

because it is i2c, and I am a beginner, it is probably slow, but I can't even pull them off fast enough at 1200 baud.

And thanks Doc, for pointing at the idiot and laughing. Wouldn't expect that from a radio operator.

Here is the message:

"This is a really long test string to show how many chars I can write to the display before it poops out.\n"

Here is how much I can display:

This is a really long test string to show how many chars I can wr

and here is all the code:

#include <Wire.h>

char message[9] = "01234567";
int index;

void setup(){
  blankDisplay();
  Serial.begin(9600);
}



void blankDisplay(){
  displayChar(' ', 7, 0);
  displayChar(' ', 6, 0);
  displayChar(' ', 5, 0);
  displayChar(' ', 4, 0);
  displayChar(' ', 3, 0);
  displayChar(' ', 2, 0);
  displayChar(' ', 1, 0);
  displayChar(' ', 0, 0);
}

void displayChar(char myChar, int myPos, byte myDisp){
  byte pos_mask;
  byte write_mask = B11000000;
  switch (myPos){
    //the pos_mask is the bitmask to XOR
    // against B11111111 to get the digit address, 0 rightmost.
    //   7 6 5 4 3 2 1 0
  case 7:
    pos_mask = B00000000;
    break;
  case 6:
    pos_mask = B00000001;
    break;
  case 5:
    pos_mask = B00000010;
    break;
  case 4:
    pos_mask = B00000011;
    break;
  case 3:
    pos_mask = B00000100;
    break;
  case 2:
    pos_mask = B00000101;
    break;
  case 1:
    pos_mask = B00000110;
    break;
  case 0:
    pos_mask = B00000111;
    break;
  }

  byte destination = (B11111111 ^ pos_mask);
  Serial.print(destination, BIN);
  Wire.begin();
  Wire.beginTransmission(0x20);
  Wire.write(0x00); // IODIRA register
  Wire.write(0x00); // set all of port A to outputs
  Wire.endTransmission();
  delay(1);

  Wire.beginTransmission(0x20);
  Wire.write(0x01); // IODIRB register
  Wire.write(0x00); // set all of port B to outputs
  Wire.endTransmission();
  delay(1);
  Wire.beginTransmission(0x20);
  Wire.write(0x12); // address port A
  Wire.write(myChar);     // value to send
  Wire.endTransmission();
  delay(1);

  //And then bounce these bit (7 and 6) to write to the char.
  Wire.beginTransmission(0x20);
  Wire.write(0x13); // address port A
  Wire.write(destination);     // value to send
  Wire.endTransmission();
  delay(1);

  Wire.beginTransmission(0x20);
  Wire.write(0x13);
  Wire.write(destination ^ B11000000); // address port A
  Wire.endTransmission();
  delay(1);

  Wire.beginTransmission(0x20);
  Wire.write(0x13);
  Wire.write(destination); // address port A
  Wire.endTransmission();
  delay(1);
}

void bright13(){
  delay(100);
  Wire.beginTransmission(0x20);
  Wire.write(0x12); // address port a
  Wire.write(B00000110);     // 13% brightness
  Wire.endTransmission();
  delay(100);

  Wire.beginTransmission(0x20);
  Wire.write(0x13); // address port A
  Wire.write(B00110000);     // Control Word
  Wire.endTransmission();
  delay(100);

  Wire.beginTransmission(0x20);
  Wire.write(0x13); // address port A
  Wire.write(B11110000);     // Control Word
  Wire.endTransmission();

}


void showMessage(){
  for(int i = 8; i > 0; i--){
    displayChar(message[i-1], i-1, 0);
  }
}


void loop(){
  if (Serial.available() > 0 ){
       for (byte i = 0; i < 8; i++){
         message[i] = message[i + 1];
       }
       char letter = Serial.read();
       message[7] = letter;
       showMessage();
       
  }
  
}

mixographer:
I get all the jokes about the snippets and everything, but I can't fit all the code in here, and most of it is all wire.transmit this and wire.end transmission that. If I put all the code in no one would read the whole thing and help me. It's more fun to be sarcastic.

I see that you did in fact manage to post your code. I also read the whole thing. And I wasn't being sarcastic. And it isn't fun to be told off by people you are trying to help. So wrong on a number of points, so far.

  Serial.begin(9600);

Since there are 10 bits to a character (8 plus start and stop bit) you can reckon on it taking 1/960 seconds for each one to arrive, that is 1.042 mS each.

So whatever you are planning to do, you better do it in under 1 mS per character or you won't keep up.

void displayChar(char myChar, int myPos, byte myDisp){
...
  Wire.begin();
  Wire.beginTransmission(0x20);
  Wire.write(0x00); // IODIRA register
  Wire.write(0x00); // set all of port A to outputs
  Wire.endTransmission();
  delay(1);
...
  delay(1);
...
  delay(1);
...
  delay(1);
...
  delay(1);
...
  delay(1);

What are those delays doing there? That is 6 mS per character (assuming the rest of the code takes no time to execute, which isn't true).

Does that device need those delays? What happens if you remove them?

64 bytes is the size of the reading buffer of the Serial. I guess the wire sending is so slow that you miss the other part of your message.

At first I recommend you to use the same baudrate approximatively for both of your communications (I2C and Serial), or even a little slower baudrate for the Serial, so your program has time to do other things.

You also could have your own buffer, larger than 64bytes and read all the Serial there is to read in this one before to send anything on the wire. But your memory isn't unlimited so don't make it too big. Bad idea anyway if you want to have more stuff in your application, you'll need this memory believe me :confused:

Hi Nick.

I took out the delay(1) calls and there is no change in the length of the array I can display.

They were probably vestiges of me trying to it bang these displays.

I do feel 90% of the questions on this forum could be answered simply with links to your site, so I do appreciate your help.

I do feel defensive here though. This morning I got a sarcastic "Really?" from I guy I actually buy products from and give money to. Like my question was too stupid to be asked. It's a bit of sport on this forum.

This morning I got a sarcastic "Really?" from I guy I actually buy products from and give money to

Name and shame then, please.

I'm sorry you feel that you are treated badly by this forum. There are quite a few posters who try hard to help Arduino users achieve the best from the technology.

I took out the delay(1) calls and there is no change in the length of the array I can display.

Hmm. No change? That's a little odd. Just to be certain, can you post your revised code? Even without your exact hardware it should be possible to reproduce the problem.

AWOL:

This morning I got a sarcastic "Really?" from I guy I actually buy products from and give money to

Name and shame then, please.

Well, I'm the one that asked "Really?", and I do not sell products or take money from anyone (except my employer), so I think OP has been sniffing too many fumes from his soldering iron.

Right. Then before this thread spirals out of control I will have to ask everyone involved to keep to the technical issues, or I will just delete it.

I have obviously become confused and frustrated. Apologies to Nick, Doc and both Pauls. I am sorry.

I need to rethink the whole ticker idea. I cannot display fast enough to empty the receive buffer, but I can display too fast to read. I have an old NYSE ticker here and it runs at 100 Baudot (5 bits). Now I think I know why.

I should see how fast I can read the ticker and THEN figure out how to send the data in a way I can read it reliably. Since I will control both ends, I think I will be able to make it work.

Thanks for all the comments,

Jimmy

mixographer:
I should see how fast I can read the ticker and THEN figure out how to send the data in a way I can read it reliably. Since I will control both ends, I think I will be able to make it work.

If you control both ends a simple solution would be for the PC only to send bytes (less than 64) when it receives a request from the Arduino. If you organize things nicely the new data will have arrived in the Serial input buffer before you need to use it so that the comms won't slow anything. Copy the data from the serial buffer to somewhere else and ask for more data. Then do whatever needs to be done with the copied data.

...R

Thanks Robin2.

I think this is a good idea. I was thinking the PC would just send a stream of data, but I think this is better. I'll let the computer work a bit harder so that the uC system can be easier.

Thanks for this.

Jimmy

Nick can give you more complete details, but the hardwareSerial.h header file found at your root directory plus:

hardware\arduino\avr\cores\arduino\

contains #define's for the serial buffer size:

#define SERIAL_TX_BUFFER_SIZE 64
#define SERIAL_RX_BUFFER_SIZE 64

I don't know how safe it is to increase these sizes, but I'm sure others here can tell you better than I can.

the code in your first post is wrong.

you copy message[1] into message [0]
you copy message [2] into message [1]
....
you copy message [8] into message [7]

and then you stick a new char into message [7]

More likely what is wrong is moving the data around at all.

I guess I'm trying to move the data like a queue. I thought I would just have message[9] and keep moving the newly read byte onto the end and the 8th one would fall off the end.

So to scroll them, I thought i was moving the byte at index 0 to index 1, the byte at 2 to 2 and so on and then adding the new byte at 0...

That may have been a giant misconception.

void loop(){
  if (Serial.available()){
       for (byte i = 0; i < 8; i++){
         message[i] = message[i + 1];
       }
       char letter = Serial.read();
       message[7] = letter;
       showMessage();
       delay(10);
  }
}

I'll try to figure out how to do it without all the shuffling. Thanks for the comments.

A nice trick to do with arrays is not to shuffle the data around, but to keep track of the index of the first value, and use that to define your starting point. There's no reason your data has to start at 0. Or, if you don't want to store your message backwards, your can use the index to store your end point.

For example, if you really want to store your string backwards and not have to shuffle the queue around everytime you get a new char, start at the end and fill it in backwards, using a variable to keep track of the first char.

const int message_length = 100;
int array_first_index = message_length;
char message_array[message_length];

Then, every time you get a new char, decrement the index variable and store the char in the new location. If the index is 0, you're full, so discard it.

void add_char(char c)
{
  if( array_first_index != 0 )
  {
    array_first_index--;
    message_array[array_first_index] = c;
  }
}

To get the length of your message, just subtract the index from the total length of the buffer. To reset the buffer, no need to clear the data, just set the index equal to the total length. That'll reset the starting point of the message.

Then, when you want to display your message:

for( int i=message_length-1; i<=array_first_index; i-- )
{
  // display char
}

Make sense?