My question is regarding the Serial library. In short, how do I know if data that is written to the serial port from the ardunio is read in from the serial port on my PC by an external application in a synchronous real time fashion?
Essentially, the arduino retrieves data from a sensor and then prints this data to the serial port, all of this in a continuous loop. My external application (matlab) also running in a loop reads the serial port and plots the data, in theory in real-time.
It seems to be running without any lag, i.e. matlab displays the data as i would expect. However, I don't really understand why: Let's say for arguments sake, the loop in the arduino is running roughly twice as fast as the loop in the matlab, the communication is then not synchronised.
My understanding is that Serial.print writes to a buffer, so I expect that if my external application is unable to read at the same rate that the ardunio is writing to this buffer, then there will be a "back-log" of data thus making the communication not real-time causing a lag in the display or something.
The baud-rate of the read in matlab matches that of the write on the arduino, which I would see is necessary if the read and write are synchronised but i don't see how it makes much difference if matlab is reading from a back-log of data in a buffer.
Have I got this completely wrong? Perhaps the serial.write is a blocking function that only executes when a device is able to read from the serial port. But in that case, I would not see the use of the buffer.
My question is regarding the Serial library. In short, how do I know if data that is written to the serial port from the ardunio is read in from the serial port on my PC by an external application in a synchronous real time fashion?
In short: you cannot because the Arduino can only check if the data is sent.
My understanding is that Serial.print writes to a buffer, so I expect that if my external application is unable to read at the same rate that the ardunio is writing to this buffer, then there will be a "back-log" of data thus making the communication not real-time causing a lag in the display or something.
It's correct that the data is written to a buffer. But that buffer is then emptied in an interrupt handler in the background as fast as possible. If the application on the PC does not read the data the information is lost. If you don't want that you have to establish a protocol where the recipient (PC) has to acknowledge the reception by some code. But if your PC is to slow to receive the data providing an acknowledgement makes it even more slow. It depends on the application and how important a complete reception of the data is if such a protocol is necessary.
But that buffer is then emptied in an interrupt handler in the background as fast as possible.
What's the limit on the speed at which the buffer is emptied? What triggers the interrupt handler? Does it relate to whatever is reading the data?
Where does the baudrate come in? is it the rate at which data is written to the buffer or the rate at which data is emptied? I would have thought the latter to make sure the application reading the data is synchronised.
So my understanding now is that, the output buffer is emptied by the interrupt and whatever was in it is sent to the serial port on my pc. When the application is ready it will read whatever is there. I guess it reads the next available data byte, and knows where the byte starts using the previous stop bit (this is where I suppose the matching baudrates are essential... ). Anyway, suppose all this time that the application was taking to read and process the data etc. before coming round to the next serial read, the arduino is constantly writing to its serial port, sticking stuff in the buffer and emptying it into the PCs serial port using the interrupt handler. Because the application on the PC isn't ready, all of this data is then getting lost!
Instead of waiting until the string has been transmitted, print() sets up a buffer for the string, and then is transmitted via interrupts one character at a time. This method allows your program to keep running, with the transmitting happening in the background.
. So essentially, each time print() is called a buffer is immediately created for this data and then interupts empty the buffer at a rate defined by the baudrate. This allows the code to continue executing even while data is being sent over serial. If another Serial.print() is called before the buffer is emptied then more is added to the buffer and it will begin to fill up to it's maximum size of 64 Bytes.
Data that you write is placed in a same-sized buffer (16 or 64 bytes). In the case of sending if the buffer fills up the code "blocks" waiting for an interrupt to send the next byte out the serial port.
i.e. When the buffer is filled any further calls to print() will block until there is enough space in the buffer for the message.
Now, if the arduino is putting stuff in the buffer faster than some external application is reading data on the corresponding serial port, then there will always be more data sent than can be received and thus data is lost.
Now, if the arduino is putting stuff in the buffer faster than some external application is reading data on the corresponding serial port, then there will always be more data sent than can be received and thus data is lost.
You have the picture of the serial port being a full-fledged RS-232 serial interface. The RS-232 defines two hardware signals (RTS and DTR) that allow the two connected devices to signal if they are ready to send or to receive. With these additional signals a PC connected to a modem (I don't know if you even know what these ancient devices are) is able to signal that receptions buffers are full and it has to stop sending additional data (same vice-versa).
The Arduino does not have such signals so it doesn't care if there is even another device connected and if there is one listening it doesn't care (it has no possibility to care) if that connected device is able to receive the data. So the Arduino sends the data and has no clue if any device is able to listen for it.
It's about the same as if you're sending Morse code using a lamp. You have no clue if anybody is seeing the light. Maybe someone sees the light but has no idea what Morse code is. As long as the other party doesn't respond you don't know if anyone received your message.
I hope that gives you some insights into serial communications :-).