I am sending data from serial to client, and it's partially working because I only get short messages delivered, while long messages (like 4000bit) get the most of them lost. when I use the softwareSerial example in the IDE to send from serial to serial it works with no data lost. I am trying to do like the example as possible with the serial to client code using client.write(mySerial.read()) but not working as expected! what am I missing?
#include <SoftwareSerial.h>
#include <Bridge.h>
#include <BridgeClient.h>
#define PORT 9000
// Make the client connect to the desired server and port
IPAddress addr(192, 168, 0, 100);
//not all pins on the yun are assignable as rx/tx
SoftwareSerial mySerial(10, 11); // RX, TX
// Define our client object
BridgeClient client;
int led = 4;
bool Connected;
void setup() {
pinMode(led, OUTPUT);
// set the data rate for the SoftwareSerial port
mySerial.begin(9600);
mySerial.println("Hello, world?");
mySerial.setTimeout(20000);
// Start Bridge
Bridge.begin();
//delay(1000);
// Start Console communications and wait for port to open:
//Console.begin();
//delay(1000);
//Console.println("Client connecting on port 9000");
Connected = false;
//delay(1000);
//connectToClient();
connectToClient();
}
void loop() // run over and over
{
while (mySerial.available()) {
client.write(mySerial.read());
//mySerial.read();
}
}
void connectToClient() {
// is a connection currently active?
if (!Connected)
{
// not currently connected, try to establish a connection
client.connect(addr, PORT);
delay(1000);
if (client.connected())
{
// send something to the client
client.println("hello server!");
digitalWrite(led, HIGH);
}
else
{
connectToClient();
digitalWrite(led, LOW);
}
}
}
It looks like this code should run on an Yun. We usually expect an Arduino UNO if the post doesn't contain an explicit Arduino model as for many problems the model is essential.
I am trying to do like the example as possible with the serial to client code using client.write(mySerial.read()) but not working as expected! what am I missing?
What exact output do you get if you send what exact content? What is the sending device? An Arduino too? If yes, please post the code you use there.
Which version of the IDE are you using? (The SoftwareSerial library recently got a major update) Which AVR core version?
I am sending the data (as a file content on one time, not in sequence) from pc using terminal Programm sscom32e. I am receiving the data as client using netcat on the same pc (Yun and PC connected with RS232-USB and are on the same network).
IDE version 1.8.9 (I mainly use Visual Basic for coding and bootloading though)
-AVR core version? I don't know where to find this
So it doesn't loose bytes but suddenly stops and seems to freeze.
BTW, the recursion you have in conectToClient() should have a limitation or be converted into a loop instead of the recursion.
The reason for the stop isn't obvious so you should add more code that prints to the console what happens. You don't have code that checks if the connection to the server still exists and reconnects if it failed.
In that case you have to describe in detail what you do and what output you get. In the above output you see all data at the beginning and a sudden stop. All data to that point was correctly transmitted and after that point no data is received anymore. If that's not the case, post the details.
I believe now that serial.read and client.write cann't work together at the same time efficiently. when client.write starts, serial.read stops and therefore data get lost. Now I am saving the serial.read in 3 arrays of each 255Byte, when they are full or I receive an end-marker I call client.write to send them and Serial stops anyway. This works but for max. 765 Byte data. I can't do more arrays to save the serial data due to the limited memory of the module, which means it's not the best solution right now. I will post the code for this later when I optimate it. the other Idea coming to my mind is to send orders to the serial to wait until client.write finishes and then continue receiving and so on. Is this possible? Is Serial that flexible? How could I communicate with Serial to wait and to continue?
I believe now that serial.read and client.write cann't work together at the same time efficiently. when client.write starts, serial.read stops and therefore data get lost.
No, the serial.read() calls just read from a buffer, so the bytes were already received at that time. SoftwareSerial of more or less current IDEs use interrupts to get the bits and the BridgeClient doesn't stop interrupts.
But it's a bad idea to write single bytes using the BridgeClient as using that single byte write for every byte written 8 bytes are sent to the linux part and at least 6 bytes are received.
Although the baudrate of the bridge is 250k you might get into troubles as you need about 140k of it just for the serial transfers and there will be some wait times. So it might be that your serial buffer fills up over time if the bytes are arriving at full speed and the read bytes aren't sent fast enough to the linux part. Once the buffer is full you'll loose some bytes.
The easiest circumvention is to read as many bytes as possible into a buffer on the Arduino side and send that complete buffer using the multibyte write version (client.write(buffer, length)). That's because the overhead is constant, so if you write a buffer of 20 bytes, 27 bytes are sent and 6 bytes are received.
I did a buffer of 4x255Byte arrays and the result is pretty good for the first 1kB received. While the bytes afterwards still have the same problem of losing alot of them. I can't do more buffer arrays due to the limited memory of the module. You think this is the best I can get of it? Only the first 1kB? Can't I save the buffer somewhere else than the 2.5kB small memory?
Here is a snippet from my last updated code. pls. tell me I am doing something wrong and I can do better.
const byte numChars = 255;//Max size of each array
char receivedChars[4][numChars]; // 4x255Byte array of arrays to store incoming serial data
boolean arrayIsFull[] = { false, false, false, false };//Is array full loaded?
boolean newData = false; // Are data ready to be printed?
size_t i = 0;// For-loops check-variable
void loop() { // run over and over
processSerialData();
sendToPrinter();
}
//To process the serial data received from RS232
void processSerialData() {
static byte ndx = 0;//Index for data arrays
char endMarker = '\f'; //Form Feed as terminator
char rc;//Current read char
while (mySerial.available() > 0 && newData == false) {
rc = mySerial.read();
if (rc != endMarker) {
for (i = 0; i < 4; i++)
{
if (!arrayIsFull[i]) {
receivedChars[i][ndx] = rc;
ndx++;
if (ndx >= numChars - 1) {//last byte in array
receivedChars[i][numChars - 1] = '\0';//terminate string at this array and mark it as full
ndx = 0;
arrayIsFull[i] = true;
}
if (arrayIsFull[3]) {//Means all arrays are full loaded now
ndx = 0;
newData = true;
}
break;
}
}
}
else {
for (i = 0; i < 4; i++)
{
if (!arrayIsFull[i]) {//Terminate the string at the last-written-to array and mark it as full
receivedChars[i][ndx] = endMarker;
if (ndx < numChars - 1)
{
receivedChars[i][ndx + 1] = '\0'; // For the rare case that the endmarker is at position 254!
}
arrayIsFull[i] = true;
break; // break the for loop
}
}
ndx = 0;
newData = true;
}
}
}
//To send data to printer after all arrays has been full loaded or an End-Marker received
void sendToPrinter() {
if (newData == true) {
for (i = 0; i < 4; i++)
{
if (arrayIsFull[i]) {//Print all full arrays
client.print(receivedChars[i]);
arrayIsFull[i] = false;
}
}
newData = false;//To start from the beginning --> new data can be received & processed
}
}
I did a buffer of 4x255Byte arrays and the result is pretty good for the first 1kB received. While the bytes afterwards still have the same problem of losing alot of them. I can't do more buffer arrays due to the limited memory of the module. You think this is the best I can get of it?
That's a rather complex buffering. Why do you have 4 buffers of 255 byte? Why not one of 1024 byte?
Do you know in what timing you receive the incoming data? What size are the blocks? If you get a constant stream on the SoftwareSerial (no pauses) you should think about changing the hardware because a software emulation of a serial interface is not suited for such a task as it wastes to much of the available computing power for such a task hardware is doing much better.
I used 4 arrays becausr Byte type is max. 255Byte. As I remember, I tried other types but there was a Problem with them.
Yes I know the timing when the data come (actually I hit a button to send data to arduino) the block size is not really the same every time but it's more or less700Bytes.
What hardware do you mean should I get? You mean a board with more memory? What do you suggest?
I used 4 arrays becausr Byte type is max. 255Byte. As I remember, I tried other types but there was a Problem with them.
But you can use integers for the indexing of arrays of type byte. Maybe I misunderstand something...
yes true. the data are already there and as I hit the button, the just go to the Arduino without pauses!
It should work if it's always about 700 bytes and then you have a pause again. Use a buffer of about 800 bytes, fill it up until you have a pause of let's say 3ms (use millis() for that) and as soon as you ran into that timeout, write the whole buffer to the bridge.
What hardware do you mean should I get? You mean a board with more memory? What do you suggest?
One of the modern MKR boards might be a better fit but given the above information I think the Yun might do the job.
I didn't mean "time blocks", but actually blocks with a remarkable end-marker which in this case the form feed '\f'.
So there is no pauses at all till the stream end.
After weeks, I just used software flow control XOFF/XON and I can't believe it's finally working! only one block with form feed at the end is buffered(XOFF) and the serial waits till it is sent to the client and then serial continues!(XON).
I can't believe that I spent so much time on such a trivial thing!
Thanks pylon! You helped me alot.
Now I am moving to implement a parallel port (yes parallel!) to the board to receive the same data and I will be then having parallel as well as serial inputs.
pylon:
But you can use integers for the indexing of arrays of type byte. Maybe I misunderstand something...
I tried int and reduced the code to only one array(accordingly no for loops needed)and didn't work!
I then just changed back <int numChars 1000> to <byte numChars 255>(just this!!) for the same reduced code and it worked! And I discovered then I don't even need a buffer of 4 arrays when I am using the flow control! Just when the only array is full I send XOFF and so on as described before... !!!!
