Hi, I´m in the middle of a project, my first big one using Arduino, and need some advice with the topic of serial communications.
I have a Processing sketch that sends a string to an Arduino UNO R3 through serial. I'm using 250000 baud rate and I'm sending a long array of data (135 four-digit values). The string, generated in Processing, has this structure:
<4095 4095 4095 ... 4095 4095>
The string is made to send RGB info to 135 LEDs (45 RGB LEDs) at 12bit resolution once per frame. The problem is that because of the array size plus the limits of the Arduino serial port speeds, the code only works when the Processing sketch is running at 20fps, and my objective is to run at at least 25 (I'm very close!).
I was told that I can improve the communication speed by sending bytes instead of a string, but I don't know where to start to adapt my current code to do that.
This is my current code:
#include "Adafruit_TLC5947.h"
#define NUM_TLC5974 6
#define data 4
#define clock 5
#define latch 6
#define oe -1 // set to -1 to not use the enable pin (its optional)
Adafruit_TLC5947 tlc = Adafruit_TLC5947(NUM_TLC5974, clock, data, latch);
char val; // Data received from the serial port
char receivedChars[674];
char arrayOfNums[135];
boolean started = false;
boolean ended = false;
int serialIn = 0;
void setup()
{
//initialize serial communications
Serial.begin(250000);
tlc.begin();
if (oe >= 0) {
pinMode(oe, OUTPUT);
digitalWrite(oe, LOW);
}
}
void loop(){
while(Serial.available() > 0) {
int incomingByte = Serial.read();
if(incomingByte == '<'){
started = true;
ended = false;
} else if (incomingByte == '>') {
ended = true;
break;
} else {
receivedChars[serialIn] = incomingByte;
serialIn++;
}
}
if (started && ended) {
int arrIndex = 0;
char * token = strtok(receivedChars," ");
while (token != NULL){
int aNum = atoi(token);
if(arrIndex <= 134){
//arrayOfNums[arrIndex] = aNum;
tlc.setPWM(arrIndex, aNum);
arrIndex++;
}
token = strtok (NULL, " ");
}
tlc.write();
serialIn = 0;
receivedChars[serialIn] = '\0';
started = false;
ended = false;
}
}
Could you please help me with a little guidance to how to start?
I don't know Processing so I can't help with that side of things.
Sending binary data will certainly result in shorter messages - 4095 can fit into an an int (2 bytes). But with binary data it is more difficult to know when the data starts and ends because the byte values used as start- and end-markers might legitimately appear in the data. That can be avoided by treating (say) byte values 253, 254 and 255 as special cases. See the code in the Original Post in this Thread. Note that there is a different version in Reply #4 in that Thread).
I like the idea of sending data in human readable form because it makes debugging easier. In my recent projects when I need extra communication speed I have been converting numbers into a sort-of base64 code. So 4095 would become
4095 / 64 = 63 (integer maths) with the remainder of 63. And then I add 48 (so that a byte value of 0 becomes 48) and 4095 encodes as the two chars "oo" (lower case O). That way I can still use < and > as start and end markers. This may sound complex but it actually decodes in about 1/10th of the time required by atoi().
Hi. Thank you for your comments. Yes, I was aware of the difficulties of working with binary data, but can it be avoided with some synchronisation? For example: sending a request when the Arduino is ready to receive and send from Processing in that moment. Or would it be even slower?
I was reading too about compressed data (I think that's what you're talking about on the second half of your post). Maybe I should try that, seems to be a smoother transition from my current code. I do not understand, though, the conversion you describe. You mean using numbers 0 to o (lower case O)? What function can I use to make the conversion? I do not understand why you add 48. Sorry, I'm new on this kind of stuff.
You could use signed int16, for example. Use positive numbers for the data values and negative numbers as commands.
-1 is 1111 1111 1111 1111 (or 0xFFFF) in binary. You could easily use this as the start mark. 0xFF00 could be a as an end mark, because the first four bits could never be 1 in your operation range.
With that approach, you can directly write the data into a storage array without further conversion. It should be (nearly) the fastest you can get. If you want to improve speed even further, you could use a custom 12 bit format and pack the data even further. To speed it up even more, it would be possible to use a simple compression like the run-length encoding, but that highly depends on the data you are transmitting.
Hi, I had to leave the project aside for a time, but I'm now into this again. I understand the code and how to make it work, but I have some questions:
Why do you divide by 64? Doesn't that implies that I'm decreasing the resolution? I understand that the ASCII table is limited, but I have, for example, 90 symbols from "$" to "~", shouldn't I try that resolution instead? (And use maybe "!" and "#" as start and end markers?)
EDIT: correct me if I'm wrong, but the decoding part of the code shouldn't be?:
And my final question is how do I separate the two symbols into highByte and lowByte?
Currently I'm sending, for example:
<A1 0f 47 ... Xi N5>
My Arduino code can detect that the values are between spaces, and it used to process those values with atoi(), but now I have to also separate the two symbols' values and store them on highByte and lowByte. I don't know how to do that.
This is the fragment of my code I should be modifying:
maurobarreca:
Why do you divide by 64? Doesn't that implies that I'm decreasing the resolution?
Dividing by a power of 2 is very easy for an Arduino. The next bigger one is 128 and I don't think there are 128 printable (and easily recognizable) characters.
Division by other numbers is VERY SLOWWWWWW.
You are correct to think that you could get a bigger real number into 3 encoded bytes if you use a divisor bigger than 64.
My Arduino code can detect that the values are between spaces
When I use my method I don't have any spaces. I know that the first 2 bytes is one item, the next 3 bytes is another item etc. And I have written functions to convert from either a 2-byte or 3-byte coded value to an integer. I don't use atoi() because it is not suitable and even if it were it is much slower.
If you don't want to bother with my encoding scheme then the parse example in Serial Input Basics shows how to extract data from a cstring.
OK, thank you very much, Robin. I'll try doing it without spaces, my only doubt is that my string is too long (135 values). I'm reducing it from aprox. 675 symbols to 270 thanks to your method (it's 40% of what I was sending before) so I guess there should be a lot less communication problems.
I understand now that you divide by 64 for optimisation reasons, sorry for my ignorance, optimisation is of great value for my project. There's no problem reducing the resolution, I don't think people can notice by eye the difference between 4095 brightness values and 2620 (I think that's the new resolution). It is still much better than 255 values.
maurobarreca:
There's no problem reducing the resolution, I don't think people can notice by eye the difference between 4095 brightness values and 2620 (I think that's the new resolution). It is still much better than 255 values.
I did not know that is what you had in mind by reduced resolution. With my system there will be no reduction in the resolution of the data. You can send 4095 just as easily as 2620. 4095 will encode into two bytes with the value 63 and 63. 2620 will encode as 40 and 60. (Note that all my numbers are in decimal).
You would then add 48 to bring the byte values into the range of convenient printable characters - so that 2620 is transmitted as 88 and 108 which is 'X' 'l'. (The second character is lower-case L)
If you need to send a value bigger than 4095 it would have to be encoded into 3 bytes.
Oh, I understand now how you store the number, I was doing something different (basically mapping 0-99 to 0-64 for each half of the four digit number). I understand now that what you were doing is dividing by 64 the whole number and then calculating the remaining.
For example:
highByte = 4095 / 64 = 63.984375 = 63 (int)
lowByte = 4095 - ( 63 * 64 ) = 63
Is this correct? I guess it is, is so much simpler that what I was doing and there's no resolution reduction...
Ok, I written my code on both ends. I know the Processing side is working and sending the string, but the Arduino side seems to not be working (I know thats a code issue and not a hardware one, the previous code worked well).
I still couldn't test it very well (it was just one try). I'll leave you the code here, my biggest doubt is the mode in which I'm reading the string, I don't know if I have to do something to read a symbol as his ASCII number (for example: H=72).
const byte numChars = 270;
int receivedChars[numChars];
int tempChars[numChars];
boolean newData = false;
void setup()
{
Serial.begin(500000);
}
void loop(){
recvWithStartEndMarkers();
if (newData == true) {
parseData();
//here goes some code that sends data to the circuit
newData = false;
}
}
void recvWithStartEndMarkers(){
static boolean recvInProgress = false;
static byte ndx = 0;
static byte startMarker = 40;
static byte endMarker = 41;
int rc;
while (Serial.available() > 0 && newData == false) {
rc = Serial.read();
if (recvInProgress == true){
if (rc != endMarker) {
receivedChars[ndx] = rc;
ndx++;
if (ndx >= numChars) {
ndx = numChars - 1;
}
} else {
recvInProgress = false;
ndx = 0;
newData = true;
}
} else if (rc == startMarker) {
recvInProgress = true;
}
}
}
void parseData() {
int i = 0;
int ledNum = 0; //LED number
byte firstByte = 0;
byte secondByte = 0;
int ledValue = 0;
while ( i < numChars ) {
firstByte = (receivedChars[i]-48)*64;
i++;
secondByte = receivedChars[i] - 48;
i++;
ledValue = firstByte + secondByte;
//here goes some code that stores the "ledValue" value on it's respective index (ledNum)
ledNum++;
}
}
The code is a copy/paste from your code, Robin, the part I'm not so sure about is if when I use Serial.read() I'm getting the ASCII number of the symbol sent or if I have to convert.
