Pages: [1]   Go Down
Author Topic: String over serial and writing byte-by-byte to 1602  (Read 1124 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 13
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm pretty new to Arduino and not the strongest of programmers. As a starter project I wanted a get my Twitter mentions over USB serial and displayed on a 1602 LCD as if they were being typed, line by line.

I've set up the circuit and written some code. It's all working nicely. Well, apart from that at present it will only display 63 bytes of data correctly and one incorrect byte as the 64th. Lowering the baud speed of the serial comms increases this numbers, as does removing the delay in which data is written to screen in the while loop (where Serial.read() is used).

I've done some experiments with just having the board read serial to a string and then rewrite the string back out and even then I can't get the required 140 characters.

Here's the code:

Quote
//TYPING TWITTER SKETCH BY WILL LUTON

#include <LiquidCrystal.h>

LiquidCrystal LCD(12, 11, 5, 4, 3, 2);

int randMin = 50; // minimum wait for next character
int randMax = 250; // maximum wait for next character
int endDelay = 3000; // wait once characters are displayed
int startDelay = 50;
int messageLoop = 0;
int charNum = 0;
String message = "Waiting for serial input..."; // Initial message
char messageOutput[]= "";

void setup(){
  LCD.begin(16,2); // set up the LCD's number of columns and rows
  LCD.blink(); // sets blinking cursor
  // for loop writes message to screen
  for (int i = 0; i < message.length(); i++) {
    if (i==15){
      LCD.setCursor(0,2);
    };
     delay(startDelay); // Wait for between min and max (simulates human type)
     LCD.write(message); // write i index of string message
   }
  Serial.begin(9600); // initialize the serial communications
};

void loop(){
  // when characters arrive over the serial port...
  if (Serial.available()) {
    // wait a bit for the entire message to arrive
    delay(100);
    //Clear LCD for new message
    LCD.clear();
    Serial.println("New input");
    // Resets message character length
    messageLoop=0;
    charNum=0;
    // read all the available characters
    while (Serial.available() > 0) {
      
      // Debug - echo serial back
      charNum++;
      Serial.write(Serial.peek());
      Serial.print(" - ");
      Serial.print(charNum);
      Serial.println("");

      // display each character to the LCD
      LCD.write(Serial.read());
      // Check if character is over 16 then move to next line
      if (messageLoop==15){
        LCD.setCursor(0,2);
      }
      // Check if character is over 32 then clear
       else if (messageLoop==31){
        LCD.clear();
        messageLoop=0;
      };
      // Increments character
      messageLoop++;
      // Wait for between min and max (simulates human type)
      delay(random(randMin, randMax));
    }

  }
  
  
};


And here's a what serial monitor looks like when running the sketch using a 140 byte lorem ispum:



Quote
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam dolor turpis, iaculis in posuere sit amet, feugiat a odio. Vestibulum metus.

Any ideas on what could be causing this? My first thoughts were how long the data is held in buffer, so they're wiped before it can be fully read. But like I say: I'm a total n00b smiley-wink

Other than that, loving Arduino.
« Last Edit: December 13, 2011, 03:42:43 pm by willmp » Logged

Seattle, WA USA
Online Online
Brattain Member
*****
Karma: 652
Posts: 50850
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
char messageOutput[]= "";
This declares an array, called messageOutput, and makes the compiler determine the size. The compiler does this by counting the number of initializers present. How many are there? How big is the array?

Code:
    // wait a bit for the entire message to arrive
    delay(100);
What is sending the data to the Arduino? Twiddling your thumbs hoping that all the data will have arrived while you were doing nothing is a recipe for disaster.

Quote
And here's a what serial monitor looks like when running the sketch using a 140 byte lorem ispum:
The buffer is 128 bytes. If you send 140, you may be overflowing the buffer with all the twiddling that you do.

Logged

New Jersey
Offline Offline
Faraday Member
**
Karma: 72
Posts: 3760
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Data read from the serial port is buffered. I think the size of that buffer varies by Arduino type, for an Uno at least, it is 128 bytes. When you send ~140 bytes of Lorum ipsum in one go, you're going to overflow that buffer and characters will get lost. Feels like your buffer size is 64.

Happily, serial communications are very slow, so as long as you can read data out more quickly than it comes in, you won't drop anything. The delays in loop make this unlikely, get rid of them.

If you're using a version of the Arduino software earlier than 1.0, Serial.print is not buffered, it blocks, meaning that it will send character by character, waiting for the UART to become free. Your print statements therefore have the ability to slow down your sketch to the point where incoming text may overflow. If getting rid of the delays doesn't solve your problem, consider reading the serial data into your own  buffer and do the prints after you have received the data.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 13
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

So I wrote this sketch:

Quote
#include <LiquidCrystal.h>

LiquidCrystal LCD(12, 11, 5, 4, 3, 2);
char inData[140]; // Allocate some space for the string
char inChar; // Where to store the character read
byte index = 0; // Index into array; where to store the character
byte writeOut = 0; // when to write out

void setup(){
  Serial.begin(9600); // initialize the serial communications
  LCD.begin(16,2); // set up the LCD's number of columns and rows
  LCD.write("Waiting...");
}

void loop()
{
  if(Serial.available()) // Read to char if data is available
  {
    delay(100); // Wait for data to arrive
    while (Serial.available()){
    inChar = Serial.read(); // Read a character
    inData[index] = inChar; // Store it
    index++; // Increment where to write next
    inData[index] = '\0'; // Null terminate the string
    writeOut=1;
    }
  index=0;
  while(writeOut==1){
    LCD.clear(); // Clear the LCD for the message
    Serial.println("[BEGIN]");
    Serial.write(inData);
    Serial.println("[END]");
    writeOut=0;
  };

}


And I now get this output:

Quote
[BEGIN]
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam[END]
[BEGIN]
sit amet, feugiat a odio. Vestibulum metus. [END]

So it seems to read the buffer to string (first 62 bytes). Then print to serial. The next 34 bytes are lost. The rest of the buffer (44 bytes) is then read over the same string and printed.

Why is it not all read in that string?
Logged

Global Moderator
UK
Offline Offline
Brattain Member
*****
Karma: 310
Posts: 26621
I don't think you connected the grounds, Dave.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

How many characters do you receive during your "delay (100);"?
Logged

"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

Fort Lauderdale, FL
Offline Offline
Faraday Member
**
Karma: 71
Posts: 6144
Baldengineer
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Note, you made your inChar[] array 140 characters.  However, you are always appending a null character.  So if you do receive 140 characters, you are writing past the end of your array.

Here's a suggestion on how I would approach this.  The amount of time it takes to receive a full tweet over serial is relatively short.  So why not receive the entire thing before you start trying to write it out to the LCD?  You'll notice I don't do a "if serial is available" because there is no need.  As long as there are characters available put them into a buffer, UNTIL an end of line delimiter is received. 

In this case I am using a newline.  To test in the serial monitor make sure you turn on the "newline" in the lower right-hand corner drop down.

Code:
char inData[141]; // Allocate some space for the string PLUS a character for \0
char inChar; // Where to store the character read
int index = 0; // Index into array; where to store the character
byte writeOut = 0; // when to write out

void setup(){
  Serial.begin(9600); // initialize the serial communications
}

void loop()
{
  while(Serial.available()>0) // fill the butter
  {
    char incomingCharacter = Serial.read();
    if ((incomingCharacter == '\n') || (index >= 140))   {   // you need SOME kind of marker you reached the end (turn this on in the monitor)
      writeOut=1;   // don't "write out" until a full tweet is received
    } else {
      writeOut=0;
      inData[index++] = incomingCharacter;
    //  inData[index] = '\0';
    }
  }

  if (writeOut==1) {   // buffer is full, now write it out.
    Serial.println("[BEGIN]");
    Serial.println(inData);
    Serial.println("[END]");
    writeOut=0;
    index=0;
  } 
}
Logged

Capacitor Expert By Day, Enginerd by night.  ||  Personal Blog: www.baldengineer.com  || Electronics Tutorials for Beginners:  www.addohms.c

Offline Offline
Newbie
*
Karma: 0
Posts: 13
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks loads James!

Have now written this and appears to be working (not tested with less than 140 chars yet):

Code:
//TYPING TWITTER SKETCH BY WILL LUTON

#include <LiquidCrystal.h> // include LCD lib

LiquidCrystal LCD(12, 11, 5, 4, 3, 2); // defines LCD

char tweet[141]; // Allocate some space for the string PLUS a character for \0
int index = 0; // Index into array; where to store the character
byte writeOut = 0; // when to write out
int typeDelay = 80; // how long to wait between each char being displayed on LCD
int endLineDelay = 1500;
String startMessage = "Waiting for serial input...";
int messageLoop = 0;

void setup(){
  Serial.begin(9600); // initialize the serial communications
  LCD.begin(16,2); // set up the LCD's number of columns and rows
  LCD.blink(); // sets blinking cursor
  // for loop writes message to screen
  for (int i = 0; i < startMessage.length(); i++) {
    if (i==15){
      LCD.setCursor(0,2);
    };
     LCD.write(startMessage[i]); // write i index of string message
   }
}

void loop()
{
  while(Serial.available()>0) // fill the buffer
  {
    char incomingCharacter = Serial.read();
    if ((incomingCharacter == '\n') || (index >= 140))   {   // you need SOME kind of marker you reached the end (turn this on in the monitor)
      writeOut=1;   // don't "write out" until a full tweet is received
    } else {
      writeOut=0;
      tweet[index++] = incomingCharacter;
    }
  }

  if (writeOut==1) {   // buffer is full, now write it out.
  writeOut = 0;
  index = 0;
  LCD.clear(); //clears any previous message and resets cursor to home
  messageLoop = 0; // resets message loop counter
  for (index = 0; index < 140; index++) {
       if (messageLoop==15){
         LCD.setCursor(0,2);
        }else if (messageLoop==31){
          delay (endLineDelay); // wait so message is readable
          LCD.clear(); // clear the screen and reset cursor to home
          messageLoop = 0;
        };
    delay(typeDelay);
    LCD.write(tweet[index]); //write character to the LCD
    messageLoop++; //increase messageLoop by 1
  }
  } 
}
Logged

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 99
Posts: 4835
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Why do you bother playing "fill the buffer" at all?

Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Fort Lauderdale, FL
Offline Offline
Faraday Member
**
Karma: 71
Posts: 6144
Baldengineer
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Why do you bother playing "fill the buffer" at all?

Since we know the maximum length of a message is more than 128 bytes, it would make sense to empty the serial buffer as quickly as possible to ensure nothing is lost.
Logged

Capacitor Expert By Day, Enginerd by night.  ||  Personal Blog: www.baldengineer.com  || Electronics Tutorials for Beginners:  www.addohms.c

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 99
Posts: 4835
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

That's right. But why the *more than 1 character* buffer? What's the need?

around and around;

If character available
{
  get the character
  if character printable
  {
    print the character
  }
  else
  {
    evaluate the character and do what's appropriate
  }
}

This whole "must buffer the line" mentality is usually just a habit from top-down type PC programming.
Occasionally it is needed but more often it is not.
Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Seattle, WA USA
Online Online
Brattain Member
*****
Karma: 652
Posts: 50850
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Occasionally it is needed but more often it is not.
Did you read what OP is trying to do. OP wants to get the whole message, and echo it out slowly, to emulate someone typing. Why is not explained.
Logged

Pittsburgh, PA, USA
Offline Offline
Faraday Member
**
Karma: 99
Posts: 4835
I learn a bit every time I visit the forum.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I must have had a mental filter on there.
Logged

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Seattle, WA USA
Online Online
Brattain Member
*****
Karma: 652
Posts: 50850
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
I must have had a mental filter on there.
No problem. Been there, done that. Got the scars to prove it.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 13
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Okay. So spent this evening making this so it works with longer strings, not just tweets, putting the LCD bit in to its own function (writeToLCD()) and commenting.

I've also changed the name of the topic so it's easier to find for people. I hope it's of help to others who had the same problem.

Code:
// TYPING SERIAL SKETCH BY WILL LUTON
// MAKE SURE "NEWLINE" IS TURNED ON IN SERIAL MONITOR

#include <LiquidCrystal.h>                                   // include LCD library

LiquidCrystal LCD(12, 11, 5, 4, 3, 2);                       // defines LCD
char serialString [513];                                     // the string for the incoming serial message - maximum of 512 bytes, plus a terminating null
int i=0;                                                     // the index for serialString
int writeOut = 0;                                            // states if or not a full string is recieved and ready to be written to LCD

void setup(){
  Serial.begin(9600);                                        // initialize the serial communications
  LCD.begin(16,2);                                           // set up the LCD's number of columns and rows
  LCD.blink();                                               // sets blinking cursor
  };

void loop(){
    while(Serial.available()>0)                              // fill the buffer
  {
    char incomingCharacter = Serial.read();
    if (incomingCharacter == '\n')  {                        // when a newline is recieved or the string hits 140 chars it's the end of the message (turn this on in the monitor)
      writeOut=1;                                            // now message is recieved write it to the LCD
      serialString[i] = '\0';                                // nullifier that signals the end of the string, so character beyond the current index are erased
      i = 0;                                                 // index for the string is reset
    }
    else {
      writeOut=0;                                           // message is not fully recieved so don't write it
      serialString[i++] = incomingCharacter;                // add char to string and increment the index
    }
  }
  if (writeOut==1){                                         // if a message is stored in the string
  writeToLCD(LCD, serialString, 80, 1500);                  // send message to custom function writeToLCD so as it can be displayed on the LCD
  delay(1500);                                              // wait user has time to read what is on screen
  LCD.clear();                                              // clear screen
  }
};


void writeToLCD(LiquidCrystal nameOfLCD, String message, int typeDelay, int fullScreenDelay){// this function writes characters to an LCD as if it had been typed. Waits when screen is full so is readable
  nameOfLCD.clear();                                        // clear the LCD ready for message
  int charsDisplayed = 0;                                   // defines new variable that counts char already written to the LCD
   for (int i = 0; i < message.length(); i++) {
      if (charsDisplayed==16){
        nameOfLCD.setCursor(0,2);                           // if first line full, move cursor to second line
      }
      else if (charsDisplayed==32){
        delay (fullScreenDelay);                            // wait so message is readable
        nameOfLCD.clear();                                  // if second line is full, clear the screen and reset cursor to home
        charsDisplayed = 0;                                 // reset numbers chars on screen
      };
      nameOfLCD.write(message[i]);                          // write char in string with index of i to the LCD
      charsDisplayed++;                                     // increase charsDisplayed by 1 as a char is now on screen
      delay(typeDelay);                                     // delay between before next char is typed
   }
  return;                                                   // return to where function was called from
};
Logged

Pages: [1]   Go Up
Jump to: