EDIT: Problem was in fact my understanding of Serial and Baud Rate. This is explained later in the thread.
Hello All,
I am currently working on a project (using Arduino v022) which takes 400 bytes of serial data, does some processing, then sends it over Ethernet to a server. My question relates to the receiving of large amounts of data over Serial.
My code which deals with the reading in of Serial data is as follows:
void getSerialData(char* dest){
emptyBuffer(dest);
// Wait for serial data to become available
while (Serial.available()<60){
delay(1);
}
char c = ' ';
while (Serial.available()>0){
c = Serial.read();
appendCharToBuffer(dest, c);
}
}
This function is periodically called in the main loop() function, and so is essentially polling for data. I do not think this is an ideal method of reading the data (keeping in mind that the data is 400 bytes long, and the serial buffer is only ~64 bytes long). I am also not so sure about using Interrupts to read the data either.
What would people suggest is the best way to read in the Serial data (please do tell me if any of my previous assumptions are incorrect). I was planning on migrating my code to ARDUINO 1.0 after I had this issue sorted, but if it is suggested that there is a nice way of doing this in ARDUINO 1.0, I may do that sooner rather than later.
What would people suggest is the best way to read in the Serial data
How you capture your serial data will depend on how the serial data itself is formatted. If the data has delimiters, then remove the delay and capture until a delimiter is reached. If the data is just a "spew" of bytes, then the delay might need to be moved to some type of serial available checking loop.
edit: wouldn't using <1 be more efficient for the delay loop?
Hard to tell what you're doing from that snippet. Do you have to get all 400 bytes into a buffer and then process them or can you work them as you get them and get rid of them on the wire? What kind of Arduino are you using?
I believe the data does have start and end delimiters, so perhaps checking for those would be a better way of doing this. I used the delay, while (Serial.available()<60), as the Arduino appeared to be emptying buffer faster than the serial data was arriving. This should not be a problem if I use delimiters.
Apologies for the whole problem not being clear; I need all of the 400 bytes of serial data before I can process it. I am using an Arduino Uno on ARDUINO 022
I will have a play and see if I can get it working reliably. As for the my eventual updating of the code to work on ARDUINO 1.0, perhaps it would then be better to use SerialEvent() together with Serial.bufferUntil()? (Am i right in saying that bufferUntil() is the Serial implementation of the Stream.findUntil()?). Any thoughts?[1]
Thanks again for your help,
Chris Socha
[1] EDIT: Problem has developed, see following post.
I now remember why I didn't use delimiting characters before! See code below:
void getSerialData(char* dest){
emptyBuffer(dest);
char c = NULL;
//wait code here
while (c != '#'){
if(Serial.available() > 0){
c = Serial.read();
appendCharToBuffer(dest, c);
}
}
}
Having timed this segment of code, it takes ~400ms for it to process 400 bytes of Serial data, which is too slow for my specification. I need to be able to read 400 bytes of serial data, do some processing on it, and send the processed data over Ethernet every 500ms (The time to process and send over Ethernet is not significant)
I also tried to see if allowing the buffer to fill before beginning reading the serial data would improve timing. The following code was placed in the code segment above //wait code here:
With just the code snippets you have posted, there is no way to help you.
Waiting for all 400 bytes to arrive is not possible, when there is only a 128 character buffer. Therefore, any waiting wastes time. Read and buffer what is available. Do something when all the data arrives, but read it as soon as it arrives.
You could possibly being sending data via ethernet while collecting the next batch of data.
What is in the 400 byte packet that requires that you read all of it before doing any processing?
What baud rate are you using?
Having timed this segment of code, it takes ~400ms for it to process 400 bytes of Serial data, which is too slow for my specification. I need to be able to read 400 bytes of serial data, do some processing on it, and send the processed data over Ethernet every 500ms (The time to process and send over Ethernet is not significant)
This may not be realistic. That last statement is questionable.
What is the serial data coming from?
Posting more information is really needed to provide any help for you.
it takes ~400ms for it to process 400 bytes of Serial data,
At 9600 (you don't say what your serial speed is), it takes just over 1ms to transmit or receive a single character - are you timing "process" time (ie the time it takes to do something with the 400 characters once they're all buffered), or receive/transmit time?
The baud rate is 19200, but previously I was using 9600, so thanks for making me stop and think for a second.
I think I'm right in saying that baud is bits/second, and with 8 bits in a byte, we can calculate the time by:
time = (bytes*8)/baud
And so...
@ 9600 baud, 400 bytes takes ~330ms
@ 19200 baud, 400 bytes takes ~170ms
(I have a feeling I've worked this out wrong!)
The serial data is being written to a JSON message for transmission over Ethernet. During this testing phase, the JSON message just holds the full 400 bytes of serial data, and a few identifiers. Later on the plan is to turn this serial data into information to be send in the JSON message, rather than just retransmitting the serial data in another format.
My timing of the Serial reading was done by taking the time at which (Serial.available() != 0), and subtracting this from the time at the end of the function.
Having timed this segment of code, it takes ~400ms for it to process 400 bytes of Serial data, which is too slow for my specification. I need to be able to read 400 bytes of serial data, do some processing on it, and send the processed data over Ethernet every 500ms (The time to process and send over Ethernet is not significant)
This may not be realistic. That last statement is questionable.
I thought I would test the Ethernet function to make sure the timing of it were as insignificant as I had first thought, and it turns out you are right, this takes ~250ms. This is not ideal, and is something which I will have to look into (The code for the HTTP put is below for your reference):
/* Attempts to do a HTTP PUT (JSON) at the url provided. */
void httpPutJson(const char* json, const char* url){
if (client.connect()){
client.print("PUT");
client.print(" ");
client.print(url);
client.print(" ");
client.println(httpVersion);
client.print("Content-length: ");
client.println(strlen(json));
client.println();
client.println(json);
client.println();
client.println();
while (!client.available()){
delay(1);
}
if (checkStatusCode()){
while(client.available()){
char c = client.read();
}
wdt_reset();
}
}
client.stop();
}
I think I'm right in saying that baud is bits/second, and with 8 bits in a byte, we can calculate the time by:
Close, but there's a two bit period per-character overhead for the start and stop bits, so an eight bit character consumes ten bit periods.
At 9600, one bit period is about 104us, so a character takes about 1.04ms to send.
Essentially the problem I was having was not fully understanding Serial and Baud Rate.
I had the impression you could speed up the way the Arduino read the serial data by waiting for the buffer to fill, but indeed the time from the first serial bit arriving to the last bit arriving was constant (as calculated in a previous post).
The specification for my project has since changed, so timing will be less of an issue. Thank you for making me understand my incorrect assumptions on how Serial functions!