I am working on a project to convert a string input into Morse code, and I'm feeling very restricted by the 64 byte size of the serial buffer. Does anyone know of a workaround for transmitting strings longer than 64 characters to the arduino without overflowing the buffer?
you can edit the source of the serial library and increase the size of the buffers to e.g. 128 bytes (use power of 2)
It will cost memory...
Rob, thanks for the suggestion. I've read that that's possible, but I was hoping for slightly more room. My ultimate goal for this project is to convert tweets (from Twitter) into Morse code, which would require space for at least 140 characters.
You might make your own circular buffer and "decant" the serial input buffer into yours. It can be any size you like.
There's an example here, also with morse output: bitlash/morse2.pde at master · billroy/bitlash · GitHub
-br
You can make it any size you like. Sizes == powers of 2 are fast as the index math which typically uses a modulo % can be done by ANDing the index which is much faster.
if you adjust the size be sure to get all buffer index math right
which version of arduino are you using BTW?
Rob, I've got an Uno R3 (is that specific enough? I'm new to this). When you say to get the buffer index math right, what exactly are you referring to?
Billroy, that sounds like an appealing solution, but could you please explain how a circular buffer works? I was a bit confused by that example.
Why not sending 140 characters in 3 pieces? What was the issue with the 64 byte buffer? Did you get some errors when you tried to send 140 bytes of message? I'm curious.
GoatFood:
I'm feeling very restricted by the 64 byte size of the serial buffer.
Do you have a tangible example of how this restriction is causing you trouble? Perhaps there are other ways around it.
The problem with sending the message in chunks is that I don't have a good way to break up the message beforehand. Because I eventually want the process to be automated, the message will have to go directly from the internet to the arduino (via serial).
I didn't get errors when sending messages larger than 64 bytes, the input just gets truncated. My understanding is that the input travels through the serial, one byte at a time, and that if the serial buffer fills up the data waiting to be transmitted is lost.
Peter, like I said my eventual goal for this project is to have it translate tweets, which can be up to 140 characters long. So I guess my 'tangible' problem is that any tweet I input would be cut off if it exceeds 64 characters.
Well, you can read about circular buffers with a little searching. But basically it's the same thing the Serial object is doing for you: you keep an array or buffer, and there is a head and a tail pointer. You push things (serial characters) in at the head and pull them out at the tail as the consuming process requires them. There is sample code around line 165 at that URL I pasted above.
You would pump characters from the serial buffer into this new buffer, and play morse out the other end of the new buffer.
It may be easier to increase the size of the Serial buffer. Or implement some flow control between the sending computer and the Arduino, perhaps like send-a-line-and-wait-for-prompt. That's how Bitlash does it. The circular buffer is there so that Bitlash can do other things while the Morse plays out in the background.
-br
GoatFood:
The problem with sending the message in chunks is that I don't have a good way to break up the message beforehand. Because I eventually want the process to be automated, the message will have to go directly from the internet to the arduino (via serial).
I didn't get errors when sending messages larger than 64 bytes, the input just gets truncated. My understanding is that the input travels through the serial, one byte at a time, and that if the serial buffer fills up the data waiting to be transmitted is lost.
Not true for the transmission of serial data, when the buffer becomes full the next serial.write command just waits (blocks program flow, also not a good thing, but maybe not as bad as losing data) until a least one more character is sent out the uart and the buffer then has space in the buffer to hold the character.
Now on reception of serial data if your sketch does not 'unload' characters fast enough from the serial buffer to keep up with the incoming characters then you will lose data as the buffer will just 'wrap around' and write newer characters where the oldest unread characters were in the buffer.
Lefty
Lefty, in that case is there a way to manually unload characters from the buffer to free up space?
GoatFood:
Lefty, in that case is there a way to manually unload characters from the buffer to free up space?
Yes, you read the them:
char data = Serial.read();
Why not just read the data quickly (relatively) into your own buffer?
Remember that Morse is very slow, you have tine to do anything you want to do with the data.
GoatFood:
Lefty, in that case is there a way to manually unload characters from the buffer to free up space?
Are you transmitting at 115,200bps already? If you receive messages faster than you can send off to PC, then there is no way. Otherwise, use as much SRAM as possible to set up a ring buffer to handle the occasional message bursts.
Arrch:
Yes, you read the them:char data = Serial.read();Why not just read the data quickly (relatively) into your own buffer?
Wow, now I'm mystified. Read actually removes from the buffer? Because that's the approach I was already taking, and it was still overflowing. Here's the excerpt of my code that deals with serial reading:
void serialEvent() {
if (Serial.available() > 0){
delay(100);
while (Serial.available()){
char inChar = Serial.read();
input += inChar;
}
stringComplete = true;
}
}
And my baud rate is set to 9600. Will setting it faster help?
GoatFood:
Arrch:
Yes, you read the them:char data = Serial.read();Why not just read the data quickly (relatively) into your own buffer?
Wow, now I'm mystified. Read actually removes from the buffer? Because that's the approach I was already taking, and it was still overflowing. Here's the excerpt of my code that deals with serial reading:
void serialEvent() {
if (Serial.available() > 0){
delay(100);
while (Serial.available()){
char inChar = Serial.read();
input += inChar;
}
stringComplete = true;
}
}
And my baud rate is set to 9600. Will setting it faster help?
No, it will make it worse. The problem is with the delay(). Your loop() will run scores of times between each serial byte, so you should be reading one at a time, and storing it in a buffer until you get a byte that signifies the end of the transmission.
Your code, as written, is saying "Oh hey I see that you want to send me some information, I'll wait a reasonable amount of time for you to finish, then I'll take a look at the data". That would be OK if the data was under the buffer amount. Because it isn't you need to adjust your code to say "I see you're sending me data. Let's see if this character signifies the end of your message. It doesn't? Ok, let's put it away with the rest of the characters you send me until you let me know when you're done."
I understand you're worried about data being received over the Arduino's serial port faster than you can process it.
Unless your sketch is doing something that prevents it from looking at the serial port for extended periods, it should be able to read from the serial port massively faster than the data is incoming. What's your sketch doing?
Peter, I'm not really concerned about the speed of incoming data, just the fact that input strings are being truncated.
Thanks for pointing that out, Arrch, I'll revise my code and update later if I run into any issues.
And thanks to everybody else for your help!