Serial - Preventing significant code/loop slowdown

Hello,

The purpose of this post is to pick peoples brains for ideas to overcome a problem I have. There will be many ways to skin the cat, but I just want some ideas to try.

I have a LCD (24x8) character LCD which talks RS232 or TTL Serial, at a fixed 19200 Baud. For this test, I am using a 644P, just because it has 2 Serial ports. I have written some crude libraries for it for large fonts (4x3, 3x3, 2x2) and I am just playing with code at the moment to determine the most efficient way to sent this thing data.

I did a simple test, write to the LCD every loop, writing a few updated variable values to the screen, taking up about 90% of the characters on the screen. I then wrote to the terminal each loop, I had the baud on that set to 19200 also, however put it up to 115200 and had the same result.

What I experienced was the terminal data got updated maybe 3 times a second (lets say that just for arguements sake). The LCD was also updated at this rate.

So it appears that the 19200 baud is slowing down the whole process. If I disconnect the screen and attach it to the computer also, I get the same result. If I then increase the baud rate, the updates then become quicker. I am not loosing data though - from what I can tell. The numbers are still flowing in sequentially (my variables are just Var++ type things each loop).

I am just wondering what is the bottleneck. Is the serial buffer filling up, and therefore it is slowing the processor down? Or is it the serial commands just take that long to execute? Or is it something else.

I am just trying to understand, and then determine the most efficient way to keep an LCD like this updated with data, while not dragging down the uC from computing other stuff.

I could just update the things that have changed on the screen, rather than writing the whole screen at once... I could write every 0.5 seconds or something to the screen, but in doing that I get fast data on the Terminal, and then it stops as the LCD is updated, and then goes fast again. I am just not sure what the bottle neck is.

Code wise, yes I could post the code, but its nothing special what-so-ever. If we ignore the crude libraries I made and just populate the LCD with data, the result would be very similar.

Something like this: (sorry I dont have the code on me at work). Hopefully you get the gist.

int Var1 = 0;
int Var2 = 123;
int Var3 = 745;
int Var4 = 12;
int Var5 = 512;


void setup()
{
  Serial.begin(115200);  //Serial to Terminal
  Serial1.begin(19200);  //Serial to LCD
}

void loop()
{
  Serial.print("Variable 1 = ");
  Serial.println(Var1);  
  Var1++;
  if(Var1 > 10000)
  {
    Var1 = 0;
  }
  
  /**********************LINE 1 of LCD**********************/
  //LCD Position 0,0
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(32,DEC);
  Serial1.print(32,DEC);
  
  Serial1.print("Variable 2 = ");
  
  //LCD Position 13,0
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(45,DEC);
  Serial1.print(32,DEC);
  
  Serial1.print(Var2);
  
  Var2++;
  if(Var2 > 10000)
  {
    Var2 = 0;
  }
  
  /**********************LINE 2 of LCD**********************/
  //LCD Position 0,1
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(32,DEC);
  Serial1.print(33,DEC);
  
  Serial1.print("Variable 3 = ");
  
  //LCD Position 13,1
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(45,DEC);
  Serial1.print(33,DEC);
  
  Serial1.print(Var3);
  
  Var3++;
  if(Var3 > 10000)
  {
    Var3 = 0;
  }
  
  /**********************LINE 3 of LCD**********************/
  //LCD Position 0,2
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(32,DEC);
  Serial1.print(34,DEC);
  
  Serial1.print("Variable 4 = ");
  
  //LCD Position 13,2
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(45,DEC);
  Serial1.print(34,DEC);
  
  Serial1.print(Var4);
  
  Var4++;
  if(Var4 > 10000)
  {
    Var4 = 0;
  }
  
  /**********************LINE 4 of LCD**********************/
  //LCD Position 0,3
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(32,DEC);
  Serial1.print(35,DEC);
  
  Serial1.print("Variable 5 = ");
  
  //LCD Position 13,3
  Serial1.print(0x5C,BYTE);
  Serial1.print(0x42,BYTE);
  Serial1.print(45,DEC);
  Serial1.print(35,DEC);
  
  Serial1.print(Var5);
  
  Var5++;
  if(Var5 > 10000)
  {
    Var5 = 0;
  }
}

Purely a learning exercise, to know what the bottleneck is and how to write code to write to the LCD without slowing the whole uC down, effecting everything else it is doing.

Thanks

The example shows the LCD writes to Serial not Serial1, I gather that's not the case in the real code.

hardware serial doesn't have any buffering for the Tx so something like

Serial.print("Variable 5 = ");

Will send the "V" straight away (and maybe the "a") then hang waiting for the UART to be free for each following character.

At 19200 maybe that's enough to slow things down, but I'm not sure. LCDs however are notoriously slow so if there is any handshaking that may be slowing things down as well.

One thing to do is up the rate on Serial1 to 115200 (disconnect the LCD) and see if things speed up. If so then that's the problem.

Personally I prefer to separate the program's printing and the serial transmitting with a ring buffer and some interrupts, but that's not how it's written.


Rob

Oops - Good spotting Rob, All the LCD Prints above should be Serial1. I will ammend that now.

Corrected the first post code - thanks Rob.

Increasing Serial1 to 115200 (when unplugged from the LCD of course, as it doesnt support that baud) does increase the speed of the uC again and therefore the printing to the Terminal too.

So from what you said, the uC does basically wait for the hardware serial?

A ring buffer eh... good old ring buffer.... Can you give me some details please? How it one setup? It is basically it sends 1 character and then goes and does other stuff for a little while, and then comes back and sends another etc - all happening so fast though that you cant really tell its doing it, and visually the LCD will still appear to update as per normal, except the other code doesnt suffer the same...

Thanks

Is there any handshaking between the Arduino and the LCD?

Or are you free to blat characters at 19200 and assume they worked?


Rob

No handshaking, just flat out 19200 and assume they all made it.

Graynomad: hardware serial doesn't have any buffering for the Tx so something like

At 19200 maybe that's enough to slow things down, but I'm not sure. LCDs however are notoriously slow so if there is any handshaking that may be slowing things down as well.

Personally I prefer to separate the program's printing and the serial transmitting with a ring buffer and some interrupts, but that's not how it's written.

This is exactly the problem, I have run into it myself. If you use the standard hardware seria libraries then writing to the serial port is a blocking operation, meaning that the code sits and waits for the write to complete before moving on.

To solve the problem, just as Rob said, you need to establish a buffer to write to. The code would put data to be sent into the buffer, then move on. Then an ISR would actually send the data. But, I'm not sure that will actually work. The ISR still needs to do the writing, and then the ISR will block. I have this on my list of issues to deal with for my own project and I'm not sure how to fix it yet. I have to do some experimenting.

The latest (unreleased) core supports asynchronous serial writes...

https://github.com/arduino/Arduino/blob/new-extension/hardware/arduino/cores/arduino/HardwareSerial.h https://github.com/arduino/Arduino/blob/new-extension/hardware/arduino/cores/arduino/HardwareSerial.cpp

[quote author=Coding Badly link=topic=59798.msg430887#msg430887 date=1304058762]

The latest (unreleased) core supports asynchronous serial writes...

https://github.com/arduino/Arduino/blob/new-extension/hardware/arduino/cores/arduino/HardwareSerial.h https://github.com/arduino/Arduino/blob/new-extension/hardware/arduino/cores/arduino/HardwareSerial.cpp

[/quote]

Nice, thank you!

look up Peter Fleury in the internet. He has open libraries (the proper kind C libraries) for a bunch of stuff, including a buffered interrupt based serial port.

I think updating the Arduino serial library with it might be quite easy too.

The ISR still needs to do the writing, and then the ISR will block

If you hang the ISR off the TX data reg empty interrupt you won't have that problem.

I'll see if I can knock up some pseudo code.


Rob

Graynomad:

The ISR still needs to do the writing, and then the ISR will block

If you hang the ISR off the TX data reg empty interrupt you won't have that problem.

I'll see if I can knock up some pseudo code.

Ah, okay, makes good sense. I took a quick look at the links you posted and was looking to see how that problem was addressed. I did see that if the TX buffer fills, then it blocks there, but that's to be expected. What else could it do short of discarding data.

It would be really cool if we could set the buffer sizes, and do it independantly of each other. Not that I am complaining at all! I appreciate all the work everyone has done.

Whoa, all those posts while I was writing, I won't bother with the code then.


Rob

you need to prevent the buffer from blocking...

when calling the funciton that will copy the data to the buffer, you can check if empty_space>needed_space. If not, return false or an error code for your software to know that the buffer is full and deal with it.

Thanks very much everyone for the replies!

Very excited to hear about the new core.

By any chance is the Ethernet library getting an overhaul too?

I will have a look at the new core files now

Thanks everyone

If i just grab these two files (HardwareSerial.cpp/h and drop them into the existing 0022 core, is there a good chance they will work and play well together?

Two things...

  1. Yes. A good chance.

  2. Dude! The point of the Arduino is to tinker. Not just with hardware but also with software. Jump in and try it! ;)

[quote author=Coding Badly link=topic=59798.msg431791#msg431791 date=1304140947]

Two things...

  1. Yes. A good chance.

  2. Dude! The point of the Arduino is to tinker. Not just with hardware but also with software. Jump in and try it! ;) [/quote]

Oh I'll try it, but this is for a production project and I was mostly asking if there were any known gotchyas. Sometimes those things can hide for a time and bite ya later.

Thank you!

WanaGo: I did a simple test, write to the LCD every loop, writing a few updated variable values to the screen, taking up about 90% of the characters on the screen. ... So it appears that the 19200 baud is slowing down the whole process.

There's more to this than meets the eye.

24 x 8 x 0.9 = 173 characters.

At 19200 baud you can send 1920 bytes a second (10 bits per character). So you should be able to send 173 characters 11 times a second. Your figures are way slower than that. This isn't really a "blocking" issue, because if you are sending as fast as you can you are limited by hardware. Blocking only applies if you are trying to do something else as well. So, say you are also sending the 173 characters to your terminal, that would halve the rate, but that is still 6.5 times a second, not 3 times a second.

Hi Nick Sorry - my 'guess' of 3 may not be 100% correct. I was going off memory and it was a few days since I did the test. It could have been as many as 6, I honestly cant remember. I should have some time to play tomorrow.

Is there an easy way to download the files from github? I followed the procedure and downloading the application, and linked my account with the application etc, however when I attempted to download the entire folder it downloaded the master copy instead of the 'New Extensions' copy. However I really dont know what I am doing on this so its all just guessing. When I did the download it downloaded about 900mb too... there is a PACK file in there which is rather large. Ideally I dont want to have to do that.