Hello there,
I need to read out a large data from my device through arduino zero via a UART connection.
Here is how my current routine goes:
I send command from Arduino Zero to host device to request data
The host device will respond with a stream of data.
Arduino Zero readout the serial buffer
Right now I'm limited by 64 bytes, or 256 bytes of data if I increase the serial buffer size.
However, the device is sending way larger bytes of data. When I connect the receiving serial port to an emmulator program, such as TeraTerm, I can see perfectly all the data sent back from my host device, but arduino zero seems to be keeping only the first 64/256 bytes of data on the buffer and I can't have the rest of the data.
Could someone tell me how I could implement such task of receving a large number of bytes?
But arduino gave me the following output:
This just in ... aaaaa
This just in ... abcdefg
This just in ... 8967846137451273549146189246354782354723547234
83741238468923658923658235
173490174823654823562
18471046123845689235623
81234618046123905461203
1823476102
hmm, I found the mistakes, the datatype of numChars and ndx were wrong, now i changed them both to 'long', but now I still have problems that it seems the output is not completely correct.
on my emulator:
and the arduino output:
This just in ... asidhasoihduiashduiqgerugiqugsduivfusdbdfiuaewghuiqewgr2gwrgbsuiewfg7uigtrbkjbdswfviwuef
wdefkjhwuieftrgh89243tkjbdfkjdbwfukjgewfuiw3tgr243fkjewvbfkewfvbg
fuiwegfr98w4y5rbkjfkjvbewfkjvewfrugewrwhewv
ewfbkjewfuivbgewfuigewf78uiewtgfwefbewf
ewfouigewf8uiwefwf bkjbewfouiwegrf2ibfewewfnkjbwff8uibfkjfkjfvb
rewfnjbrewfoiwfh8wfnbewf bbfb
ewfnikoewnbhfoewfnbjbfkfb
oidsnfndfslkfhowfhe
as you can see the data is not completely the same...
<Arduino is ready>
This just in ... iakjdfoiahfuiwheri2ghw9vbguisbdv9082478y5943vbuidwsbug
This just in ... 1111111111
2222222222
3333333333
4444444444
5555555555
6666666666
7777777777
8888888888
9999999999
0000000000
9999999999
8888888888
7777777777
6666666666
5555555555
4444444444
3333333333
2222222222
1111111111
2222222222
3333333333
4444444444
5555555555
6666666666
7777777777
8888888888
9999999999
0000000000
9999999999
8888888888
7777777777
6666666666
5555555555
4444444444
3333333333
2222222222
1111111111
This just in ... 11111111111111111111
22222222222222222222
33333333333333332
11111111111111111111
So I have 3 data in the memory, and you can see that the first two data seems to be captured correctly, but a large bunch of data at the end of the 3rd data is missing and also corrupted. Which seems to be a similar problem with my previous post that a section of the data was missing and corrupted.
In my code, I gave a "const long numChars = 5000;" which theoretically should be able to handle the amount of data I have now, but it doesn't seem.
Could you please help me with this?
thank you very much
This just in ... aaaaaaaaaaaaaaaaaaaa
ssssssssssssssssssss
dddddddddd
ffffffffff
gggggggggg
hhhhhhhhhhhhhhhhhhhh
jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
kkkkkkkkkkkkkkkkkkkk
llllllllll
00000000000000000000
This just in ... 111111111143444444444
44444444446666666666
This just in ... 0000000000
9333
In the beginning I thought I reached certain buffer size limit, but in this second trial, I captured much less bytes of data in compared to 1st trial, so I believe there is some other mechanism playing the magic...
I need to send a command "+++" to interrrupt my host device before requesting a data. I have now decreased the delay to 500. And no improvements yet.
In any case, this for loop should only be executed once, in my opinion, since the global variable 'i' has increased to 1, therefore I think this should have no influence on the following recvWithStartEndMarkers() function?
thanks
soleilsword:
I need to send a command "+++" to interrrupt my host device before requesting a data.
I suspect you only need to send that once for every message. And DO NOT read data outside recvWithStartEndMarkers().
Keep in mind that the function recvWithStartEndMarkers() may need to be callled several times to get the whole of a single message.
The way you are doing things it seems to me you are sending the "+++" in the middle of receiving a message.
I suggest you only send the "+++" if (newData == true) and before you set it back to false. That way you will only be sending the "+++" after a complete message is received.
Dear Robin,
the "+++" and "rdlfind 2 -3 -1" are the commands to the device, which trigger the device to send back data. Since they're in a for loop which executes only once, and the reception of data will occur only after these two commands, I don't think this will cause interference with the reception of data. or am I wrong?
thanks.
for (k;k<1;k++)
{
Serial2.println("+++");
Serial2.println("rdlfind 2 -3 -1");
}
I think I might find the issue.
The current way the code is organized, it reads the incoming date from the serial buffer, and writes the captured data to another serial buffer.
When the first data (between <>) is captured by arduino, it immediately print to teraterm, while the second data continues to fill the serial buffer.
I suspect if the first print takes too much time before finishing, and if the second data is larger than the serial buffer size, overflows of data will happen before they are read.
is this a possible explanation?
Now I just read once per data and write to my serial, and I achieved 1kByte of data correctly.
soleilsword:
Since they're in a for loop which executes only once, and the reception of data will occur only after these two commands,
It seems a strange way to get something to happen once. Why not use IF ? And using the name 'k' for such an important control variable is a recipe for late nights and gallons of coffee when you come back to the program in 6 months time. Why not call it messageRequested and begin with it set to false?
And it seems to me that you could only ever receive a single message that way - in your earlier examples I thought the problem was with the 3rd or 4th message.
I suspect if the first print takes too much time before finishing, and if the second data is larger than the serial buffer size, overflows of data will happen before they are read.
The way the code is written it will only send data after a complete message is received, so sending should not interfere with receiving.
Unless, of course, the sender starts sending the second message without being asked - but i presume that does not happen unless you send another "+++"
I should say that I have no experience of the Zero so I can't say if there are any issues that may be particular to it. However I would expect it to perform better than an Uno or Mega.
yeah, I just changed 'k' to 'msgNr' for clarity, thanks for the suggestion. please see below my latest message, with the sender request data from the receive one message after another message.
In my original code, in fact, the sender starts sending the second message without being asked... in my code I specified the number of requested messages ... and I realized that there was a conflict in that way after trial and error.
the reason I sent "+++" is to ensure the next message is correctly sent, currently whenever I send "+++", I see on my emulator I actually send "v++", so if I directly send the actual message, the receive can't understand a thing becuase it's a wrong message....
so do you have any clue why I couldn't get the first message correct?
thank you very much
#include <Arduino.h> // required before wiring_private.h
#include "wiring_private.h" // pinPeripheral() function
Uart Serial2 (&sercom1, 11, 10, SERCOM_RX_PAD_0, UART_TX_PAD_2);
const int numChars = 5000;
char receivedChars[numChars];
boolean newData = false;
boolean printDone = false;
short msgNr = 0;
void SERCOM1_Handler()
{
Serial2.IrqHandler();
}
void setup() {
// initialize serial communication at 115200 bits per second: Serial1 is pin 0 and 1;
Serial1.begin(9600);
Serial2.begin(9600);
Serial1.println("<Arduino is ready>");
// Assign pins 10 & 11 SERCOM functionality
pinPeripheral(10, PIO_SERCOM);
pinPeripheral(11, PIO_SERCOM);
}
void loop() {
//send next command only if previous requested data has been completely captured and printed out, indicated by printDone == false
//this loop request the last 'k' record of data from ATM-915, and print out them one by one
if( (msgNr ==0) || (printDone == true) && (k<10) ) {
printDone = false;
Serial2.println("a");
Serial2.print("rdlfind 2 -");
Serial2.print(msgNr +1,HEX);
Serial2.print(" -");
Serial2.println(msgNr +1,HEX);
msgNr++;
}
recvWithStartEndMarkers();
showNewData();
}
void recvWithStartEndMarkers() {
static boolean recvInProgress = false;
static int ndx = 0;
char startMarker = '<';
char endMarker = '>';
char rc;
// if (Serial2.available() > 0) {
while (Serial2.available() > 0 && newData == false) {
rc = Serial2.read();
if (recvInProgress == true) {
if (rc != endMarker) {
receivedChars[ndx] = rc;
ndx++;
if (ndx >= numChars) {
ndx = numChars - 1;
}
}
else {
receivedChars[ndx] = '\0'; // terminate the string
recvInProgress = false;
ndx = 0;
newData = true;
}
}
else if (rc == startMarker) {
recvInProgress = true;
}
}
}
void showNewData() {
if (newData == true) {
Serial1.print("This just in ... ");
Serial1.println(receivedChars);
printDone = true;
newData = false;
}
}
I am always wary of complex IF statements - especially mixing and and or
I can't get away from the feeling that yours is a problem of thinking rather than programming. I'm still not at all clear how the relationship between your Arduino and the other device should be managed.
I don't know if you have said what is the device that is sending the data. Post a link to its datasheet or user manual where the interface is defined.