Hello, I am using an arduino Mega to receive serial bytes from a device. I send to device a command and I should receive as response about 3000 bytes by serial port, and I need to store this bytes in an array.
The baud rate is 115200.
Because of the buffer size I only receive 64 bytes. So I was wondering if it's possible to receive the complete data and how achieve it.
Thank you for your help.
AmbarX:
Hello, I am using an arduino Mega to receive serial bytes from a device. I send to device a command and I should receive as response about 3000 bytes by serial port, and I need to store this bytes in an array.
The baud rate is 115200.Because of the buffer size I only receive 64 bytes. So I was wondering if it's possible to receive the complete data and how achieve it.
Thank you for your help.
You can receive ALL the data if you remove each byte as soon as it arrives. Please study: Serial Input Basics - updated - Introductory Tutorials - Arduino Forum.
Paul
Have a look at the examples in Serial Input Basics - simple reliable ways to receive data.
In the example the max size of the message is 32 chars but you can change the value in numChars to anything you want. If you need more than 255 bytes just make sure to define the relevant counting variables as int rather than byte.
...R
The problem is that the received data doesn't have an end character, only has two start bytes. The received data are hex numbers.
I usually receive 10 data bytes from device, but when I send the command I should receive about 3000 bytes, the data length is included in the received bytes (bytes 3 and 4).
Robin2:
Have a look at the examples in Serial Input Basics - simple reliable ways to receive data.In the example the max size of the message is 32 chars but you can change the value in numChars to anything you want. If you need more than 255 bytes just make sure to define the relevant counting variables as int rather than byte.
...R
I changed my code, and now I am losing the normally received data (the 10 bytes data). I should receive 10 bytes and then 10 bytes for event, but now the second 10 bytes are missing.
(I have not tried yet to send the command to receive the almost 3000 bytes in this code).
const int numChars = 3000;
byte receivedChars[numChars];
const int numCharAlarm = 10;
byte receivedAlarm[numCharAlarm];
boolean newData = false;
byte message[] = {0xAA, 0x75, 0x56, 0x56, 0x56, 0x56, 0xF5, 0x01, 0x00, 0x6D};
void setup() {
Serial.begin(115200);
Serial.println("<Arduino is ready>");
}
void loop() {
recvWithStartEndMarkers();
showNewData();
}
void recvWithStartEndMarkers() {
static boolean recvInProgress = false;
static int ndx = 0;
byte startMarker = 0xAA;
byte rc;
while (Serial.available() > 0 && newData == false) {
rc = Serial.read();
if (recvInProgress == true) {
if (ndx < numCharAlarm) {
receivedAlarm[ndx] = rc;
ndx++;
}
else {
recvInProgress = false;
ndx = 0;
newData = true;
}
}
else if ((rc == startMarker)&&(recvInProgress == false)) {
recvInProgress = true;
receivedAlarm[ndx] = rc;
ndx++;
}
}
}
void showNewData() {
if (newData == true) {
for (int i = 0; i<numCharAlarm; i++){
Serial.print(receivedAlarm[i],HEX);
Serial.print(" ");
}
Serial.println();
newData = false;
}
}
AmbarX:
I usually receive 10 data bytes from device, but when I send the command I should receive about 3000 bytes, the data length is included in the received bytes (bytes 3 and 4).
So there is a protocol with a header and a payload. Read the full header so you know how many bytes are in the payload of the message and next read that number of bytes.
Can you provide complete details of the protocol?
The below can be a framework.
First you can define a struct that can hold all relevant information of a received message.
struct MESSAGE
{
byte startBytes[2];
uint16_t length;
byte payload[1000]; // 1000 max
};
You can add additional fields and/or adjust to your needs; I had to limit the size of the payload as I'm using an Uno.
Next you can declare a message variable
// message
MESSAGE msg;
And some additional variables and constants
// size of the header in the message
const uint16_t headerSize = sizeof(msg.startBytes) + sizeof(msg.length);
// expected start bytes
const byte startBytes[2] = {'a', '5'};
// receive timeout in milliseconds
const uint16_t rxTimeout = 5000;
If you add fields in the struct, you will have to recalculate the headerSize. I picked two start bytes that could be typed in serial monitor; adjust to the real ones. I've also added a timeout variable to be able to detect interruption, e.g. when somebody pulls the cable out or when the sending device stops working.
There are a few possible error conditions that I could see
Incorrect start bytes
Incorrect length which might result in buffer overflow
Timeout error
Checksum/crc error (if your message contains that type of information)
I've therefore created an enum type with defined errors that can occur; it does not only hold error codes but also the possible status of the receive 'operation'.
enum RXSTATUS
{
RX_COMPLETE, // message complete
RX_INCOMPLETE, // message not complete yet
RX_ERR_START, // startbytes mismatch
RX_ERR_TIMEOUT, // timeout error
RX_ERR_SIZE, // payload bytes will not fit
RX_ERR_CS // checksum error
};
Next you can write a function to read a message; the basics
/*
read message on serial port
Returns:
status of operation
*/
RXSTATUS readMessage()
{
return RX_INCOMPLETE;
}
And in loop() you can call this function as show below
void loop()
{
RXSTATUS status = readMessage();
if (status == RX_COMPLETE)
{
// complete message received
}
else if (status != RX_INCOMPLETE)
{
// error handling
switch (status)
{
}
}
else
{
// message not complete yet
}
}
Now we can start filling in readMessage(). The function uses a byte pointer that will allow you to treat the message variable as an array of bytes. It also uses an index variable in the same way as Robin's code does to indicate where the next byte must be stored in the 'array'. You will also need a variable to hold the extracted payload length and the time that the last byte was received.
In the first step, the code will check if a timeout occured
/*
read message on serial port
Returns:
status of operation
*/
RXSTATUS readMessage()
{
// pointer to message
byte *ptr;
// index where to store received byte
static uint16_t index;
// extracted payload length from received bytes
uint16_t payloadLength = 0;
// when last byte was received
static uint32_t lastByteTime;
// setup pointer to point to beginning of message
ptr = (byte*)&msg;
// check for timout if we already received something
if (index != 0 && millis() - lastByteTime > rxTimeout)
{
// cleanup
index = 0;
return RX_ERR_TIMEOUT;
}
In the next step, the code checks if serial data is available and store it in the 'array'. If a byte is received, the timeout is 'reset'
// if something to read
if (Serial.available() > 0)
{
lastByteTime = millis();
ptr[index] = Serial.read();
//Serial.print(ptr[index], HEX);
index++;
}
Next the code checks if the complete header is received.
// if header complete
if (index >= headerSize)
{
With the header in place, the code can check for correct start bytes and extract the payload length
// check if we have correct start bytes
if (ptr[0] != startBytes[0] || ptr[1] != startBytes[1])
{
// cleanup
index = 0;
return RX_ERR_START;
}
//Serial.println(payloadLength, HEX);
// determine payload length
payloadLength = ptr[2] * 256 + ptr[3];
//payloadLength = (ptr[2] - '0') * 256 + ptr[3] - '0'; // this is a hack to be able to use serial monitor as input device
// check if payload will fit
if (payloadLength > sizeof(msg.payload))
{
// cleanup
index = 0;
return RX_ERR_SIZE;
}
Note that the length bytes might be reversed in the data coming from your device; in that case you have to swap the ptr[2] and ptr[3] in the calculation of payloadLength.
Next the code checks if the message is complete. If so, reset the index variable and indicate to the caller that a complete message is received. The below also prints the message for debugging purposes.
// if message complete
if (index >= headerSize + payloadLength)
{
// reset index for next message
index = 0;
// print received message for debugging
Serial.print("message: ");
ptr = (byte*)&msg;
for (uint16_t cnt = 0; cnt < sizeof(msg); cnt++)
{
if (*ptr < 16) Serial.print(0);
Serial.print(*ptr, HEX);
Serial.print(" ");
ptr++;
}
Serial.println();
return RX_COMPLETE;
}
}
The full function
/*
read message on serial port
Returns:
status of operation
*/
RXSTATUS readMessage()
{
// pointer to message
byte *ptr;
// index where to store received byte
static uint16_t index;
// extracted payload length from received bytes
uint16_t payloadLength = 0;
// when last byte was received
static uint32_t lastByteTime;
// setup pointer to point to beginning of message
ptr = (byte*)&msg;
// check timout if we already received something
if (index != 0 && millis() - lastByteTime > rxTimeout)
{
// cleanup
index = 0;
return RX_ERR_TIMEOUT;
}
// if something to read
if (Serial.available() > 0)
{
lastByteTime = millis();
ptr[index] = Serial.read();
//Serial.print(ptr[index], HEX);
index++;
}
// if header complete
if (index >= headerSize)
{
// check if we have correct start bytes
if (ptr[0] != startBytes[0] || ptr[1] != startBytes[1])
{
// cleanup
index = 0;
return RX_ERR_START;
}
//Serial.println(payloadLength, HEX);
// determine payload length
payloadLength = ptr[2] * 256 + ptr[3];
//payloadLength = (ptr[2] - '0') * 256 + ptr[3] - '0'; // this is a hack to be able to use serial monitor as input device
// check if payload will fit
if (payloadLength > sizeof(msg.payload))
{
// cleanup
index = 0;
return RX_ERR_SIZE;
}
// if message complete
if (index >= headerSize + payloadLength)
{
// reset index for next message
index = 0;
// print received message for debugging
Serial.print("message: ");
ptr = (byte*)&msg;
for (uint16_t cnt = 0; cnt < sizeof(msg); cnt++)
{
if (*ptr < 16) Serial.print(0);
Serial.print(*ptr, HEX);
Serial.print(" ");
ptr++;
}
Serial.println();
return RX_COMPLETE;
}
}
return RX_INCOMPLETE;
}
The below loop demonstrates how it can be used.
void loop() {
RXSTATUS status = readMessage();
if (status == RX_COMPLETE)
{
Serial.print("start bytes: ");
for (uint8_t cnt = 0; cnt < sizeof(msg.startBytes); cnt++)
{
if (msg.startBytes[cnt] < 16) Serial.print(0);
Serial.print(msg.startBytes[cnt], HEX);
Serial.print(" ");
}
Serial.println();
Serial.print("payload length: ");
Serial.println(msg.length, HEX);
Serial.print("payload: ");
byte *ptr;
ptr = msg.payload;
for (uint16_t cnt = 0; cnt < sizeof(msg.payload); cnt++)
{
if (*ptr < 16) Serial.print(0);
Serial.print(*ptr, HEX);
Serial.print(" ");
ptr++;
}
Serial.println();
}
else if (status != RX_INCOMPLETE)
{
// error handling
switch (status)
{
case RX_ERR_START:
Serial.println("startbytes incorrect");
break;
case RX_ERR_TIMEOUT:
Serial.println("a timeout occured while receiving a message");
break;
case RX_ERR_CS:
Serial.println("message corrupted; checksum error");
break;
case RX_ERR_SIZE:
Serial.println("payload length exceeds buffer size");
break;
default:
break;
}
}
}
The above code is non-blocking. This means that when you send a message to your device, you will have to keep track of that. The basics of loop() could look like
void loop()
{
static bool awaitingReply = false;
if (awaitingReply == false)
{
// send your message to device
...
...
// indicate that we're waiting for a reply
awaitingReply = true;
}
else
{
// read message and handle as shown in previous post
...
...
// if message received or error
if (...)
{
awaitingReply = false;
}
}
}
My approach is not to "handle" the message length while receive.
Since you dont have terminating byte, just receive and fill your array as long as there are data in buffer. Every time (after receive start) you find an empty buffer, just add a delay (2-5 char @ 115200) and retry. if this delay passes and buffer is still empty it means reception end.
then you can check if message length equals bytes received so to assume a "valid message"
AmbarX:
The problem is that the received data doesn't have an end character, only has two start bytes. The received data are hex numbers.
Please post a link to the document that defines the protocol.
Have you tried using the second example (receiveWithEndMarker) and pretending that your start-marker is the end-marker? That often works fine - you just lose the very first message
...R
Continuing post #7 above:
If you handle message length bytes during receive, and read errorneously, then at the end you leave bytes unread in buffer (or you stay in loop) and you have to re-sync to begin of frame. How? by emptying the buffer again and again, until message end.
Thats the same as proposed, one shoot.
kellykporn:
Thats the same as proposed, one shoot.Same problem
!
?
demkat1:
!
?
Can you not recognize (and report) Spam ?
...R
Robin2:
Can you not recognize (and report) Spam ?...R
there was a post between 9 and 10 which was reported as porn. Post was removed, so this #10 seems "hanged".
A moderator to remove it . thank you
edit
reemove all posts #10 and after, including this one