Serial interrupts

Projects Programming
1

I'm new to arduino, although I have written extensively in PicBasicPro. I have a project that involves data communications, and I need to process 4 simultaneous RS-232 channels at 115K baud. I figured that I would use a TEENSY 3.2, because it runs fast and has 3 hardware serial ports. My question is- are all the hardware ports interrupt- driven? What about Software Serial? Are these ports polled, and checked once per loop, or do they run entirely in the background?

If a serial char comes in does the ISR grab it and immediately move on, or does it wait for a timeout (to see if another char is coming) before moving on?

With 4 ports running 115K baud - one of them in software, I don't want to miss any data.

And is the serial.write function interrupt-driven?

In PicBasic, I wrote all my ISR's, and knew exactly what was going on. In Arduino, so much happens in the library functions, and interrupts are seldom mentioned.

2

No idea about the Teensy, but in Arduinos in general

The hardware serial ports are interrupt driven (both TX and RX). Received data gets copied in a software buffer that you can read later, data to be send gets copied into a software buffer (except for the first byte when the transmit register is empty, that will be placed straight into the transmit register).

I've never seriously analysed the SoftwareSerial, but as far as I know the receive is interrupt driven, no idea about the transmit. It however might block for the time that it takes to receive a byte (startbit, stopbit, 8 databits, parity(?)).

There are alternatives to SoftwareSerial (AltSoftSerial and NeoSWSerial); no idea if they work on a Teensy.

FYI, a Mega has 4 serial ports :wink:

3

SoftwareSerial is very slow. Use HardwareSerial whenever you can. Also, in practice, you can only use one instance of SoftwareSerial.

A Mega would have no trouble reading 4 HardwareSerial ports but I have no experience of Teensies.

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data. You could make a copy of the function (and its data variables) for each Serial port.

...R

4

I have been reading, but I must be a slow learner. A couple of questions:

In the command

c = serial.read()

Does the READ of the buffer clear the buffer of one char? Or does the buffer remain intact after that line.

My purpose is that I need to receive a variable-length packet. The packet always starts with 0xFE. The second byte of the packet is the data length (which is the packet length - 8). All data is binary.

All the examples show receiving the bytes between a start and a stop char, or a fixed number of bytes. Or for a certain length of time before a timeout.

So, I need to check for a 0xFE, and when I see that, read the next byte (data length) and then read the rest of the variable length packet (data length + 8) into an array, and finally, extract the CRC, which is the last two bytes of the packet.

I can't find many examples.

5

After a read, the byte is removed.

Just wait for the FE, next wait for the length indication, save it and next read N bytes. Although text based, you can apply the principles in Robin2's updated serial input basics thread to achieve non-blocking reading of the serial data.

6

CharlesLinquist:
So, I need to check for a 0xFE, and when I see that, read the next byte (data length) and then read the rest of the variable length packet (data length + 8 ) into an array, and finally, extract the CRC, which is the last two bytes of the packet.

You should be able to do that with a small change to the 3rd example in Serial Input Basics

I suggest you watch for the start marker, read the size byte, gather all the message bytes and then declare newData = true. After that you can examine the CRC bytes.

Why is the packet length = data-length + 8?

...R

7

My problem is that I keep trying to think the way I would do it in PBP.

The packet length is the data length + eight bytes because the packet contains a header and CRC that are not included in the data length.

Would something like the following work?

boolean newData = false;

void setup() {
Serial.begin(115200);
}

void loop() {
recvSerialPacket();

....

}

void recvSerialPacket() {
static boolean recvInProgress = false;
static Boolean gotlength = false;
static byte ndx = 0;
char startMarker = 0xFE;
char packetlength = 0;
char rc;

while (Serial.available() > 0 && (newData == false) && done = false {
rc = Serial.read();
if (rc == 0xFE) && !gotlength {
packetlength = Serial.read();
gotlength = true;
}

packetdata [ndx]= serial.read();
ndx++;

if ndx == packetlength + 8 {
done = true
}
}

8

I would not do it like that. What if the length byte hasn't been received yet? Serial.Read will return -1 which will make an interesting length :slight_smile: In the while loop you should only do a single Serial.Read.

Remember that serial communication is slow compared to the processor speed.

When posting code, please use code tags. Please edit your post and

Type
** **[code]** **
before your code
Type
** **[/code]** **
after your code

9

This is going in the right direction but it needs more work

while (Serial.available() > 0 && (newData == false)  && done = false {
        rc = Serial.read();
          if (rc == 0xFE) && !gotlength  {
              packetlength = Serial.read();
              gotlength = true;
          }

You don't need both "done" and "newData" - just use one of them

When you get the start marker you need to set a variable - let's call it waitingForLengthByte = true as you might need several calls to the function before the next byte arrives.

When you receive the length byte you can set it back to false and save the length +8 to a variable called (say) bytesStillToCome which can then be decremented as each byte is received.

Hope this makes sense. I suspect when it is all written down some streamlining may be possible.

...R

10

I'll continue to work on it. But even though serial is slow compared to processing speed, I have to deal with 4 serial ports, and - in the end, do a lot of other tasks as well. The data can come in to all 4 ports at once. Two ports are full-duplex.

The packets must be checked for CRCs before they are re-sent out another port, so I have to buffer the packets, I can't just send them byte-by-byte as I receive them. In some cases, I have to modify the contents of the packets, re-compute the CRCs and send.

11

CharlesLinquist:
I have to deal with 4 serial ports, and - in the end, do a lot of other tasks as well.

As you have given no idea of how much processor time is required for the other tasks ....

How many bytes per second (averaged over, say, 5 minutes) arrive at each serial port?

Get the code working reliably to receive the data.
Then write a function to verify the CRC.

...R

12

if (rc == 0xFE) && !gotlength  {Oops

13

To Robin:

About 2,000 bytes of data is sent once per second from two sources. Those sources are fed into ports "A" and "B".

The data from both sources is constantly validated for correct CRCs. Three consecutive "good" packets must be received before a source is validated. If 5.5 seconds elapse with no good packets, that source is UN validated.

Buffered data from a validated source is sent out port "C". If Port "A" is validated, it is always used as the source. But if Port A is not validated, and port B is validated, Port "B" is used as the source of the data that is sent to port "C". If neither port (A or B) then a "dummy" packet is generated and sent out port "C". It will be necessary to send from port C while receiving from the other sources.

Port "C" is constantly ACKing the received data. The data sent by port C is sent to BOTH port A and Port B.

Port "D" is receiving 6 bytes every 10 milliseconds. It is OK if these packets are occasionally dropped.
The data coming in on Port D is inspected and compared against certain variables.
If certain conditions are met, the processor must generate a sequence of packets send those out port "C" -while simultaneously blocking the input from ports "A" and "B".

If the data from Port "D" reports all OK (based on data), then the packets coming in Ports A (or B) are again passed, unchanged to port C.

Ports A, B and C are bidirectional. Port D is receive-only.

Port A is actually running at 57600 baud, all the others are 115,200 baud. So data sent port A <-> port C has to have its baud rate converted in both directions.

14

I hope this is closer. The idea is to ignore bytes until a 0xFE is received. The next byte is the length byte. Then the rest of the packet (which has a length equal to the length byte +6 (the total packet length is the length byte +8, but if you subtract the 0xFE and the length byte, you are left with length byte +6)

This code uses a software serial routine.

void loop(){
   while (altSerial.available()){
      if (index_1 <= datalength +6){
       c = altSerial.read();
          dat1[index_1] = c;
       if (dat1[0]== 0xFE){
          index_1 ++;  // increment only if we have seen a start byte
          if (dat1[1] > 0){  // something has filled this value, since it was initialized to zero.
          datalength = dat1[1];
      }
      }
      }
      }
      }
15

As the above is not a complete code, it's a little difficult to judge but it's looking promising. One warning regarding the comment "// something has filled this value, since it was initialized to zero.". This will only be the case for a global variable and will only be the case the first time; so make sure you wipe that variable clean before using it a second time.

By the way, you lost me when you started talking about ACKing in relation to port "C". Do you mean that port "C" will receive an ACK or NACK after it has send data and that that needs to be passed back to port "A" and port "B"?

Lastly, the beauty of software development for Arduinos is that you can test partial code it straight away; you might want to use a decent terminal program that supports hex so you can type FE and it will send the binary value 0xFE.

16

CharlesLinquist:
About 2,000 bytes of data is sent once per second from two sources. Those sources are fed into ports "A" and "B".

Does this mean that the Mega is receiving 4000 bytes per second? Or is each stream sending 1000 per second?

The data from both sources is constantly validated for correct CRCs. Three consecutive "good" packets must be received before a source is validated. If 5.5 seconds elapse with no good packets, that source is UN validated.

I find this confusing when read in conjunction with the previous paragraph I have quoted. In 5.5 seconds over 10K of data would have arrived (or maybe over 20k). A Mega has nowhere to store all that.

And, separately, do you mean that you are keeping a rolling count of successful CRC verifications and if, at any time the last 3 are NOT ALL valid then that input stream is downgraded?

OR, is the test for 3 consecutive valid CRC checks only done once.

Please tell us what this project is for. It will make it easier to help.

I probably have other questions, but that's enough for now.

...R

17

Yes, the data comes in either port A or B and is sent to port C. Every time that port C gets a good packet (it checks for CRCs also), it sends a short ACK (about 10 bytes) back to both ports A and B for transmission.

As you may have guessed, this is a redundant communications link that uses two different methods to send data. Both ends SEND using both communications methods, but packets are received only from a validated "good" source - so (in theory) it is possible to send using one method and receive using the other. The failover should be nearly seamless. and transparent to the user. There will be a TEENSY at both ends of the link. The devices connected to the TEENSY are intelligent and capable of retries.

The downside is that the data will often come into both A and B ports of the TEENSY simultaneously. It has to be able to handle that, while dealing with port C.

One of the two TEENSIES will also be responsible for receiving data from port "D" at 115K baud. It will have to examine two bytes of a 6 byte packet (which also starts with 0xFE), and comparing that to two values. Depending on the outcome of those comparisons, the TEENSY may have to ignore both ports A and B and internally generate its own packets to send out port "C".

I chose a TEENSY3.2 over the ATMEGA because the TEENSY runs at 96MHz, has plenty of code space, is very small, not expensive ($19), and has 5V tolerant I/O on most pins.

Since the transmission methods will be radio, there can be fade, multipath, blockage, etc. Whenever a packet is received and validated, it can be sent immediately out port C. If a CRC is bad, that packet can be thrown away. Packets are sent at a rate of one "burst" per second. Each burst can contain multiple packets. But the duty cycle never exceeds about 25%. That is why reception at 115Kbaud and retransmission at 57600 baud is possible. If a period of 5.5 seconds passes with no valid CRCs, that input (A or B) is invalidated, and the other port is chosen (assuming it has valid CRCs). The 5 or 6 second "gap" in information is tolerable, since most information is retransmitted every few seconds.

The purpose:

I build drones, and I'm using both 915Mhz and 4G LTE for control and telemetry.
I need a communications method that works nearly everywhere. 4G service is generally the best, but there are places (in the mountains for example) where there is no coverage. I currently use a Raspberry pi 3 at both ends to handle the 4G link and SSH encryption. All data is sent using MAVLINK protocol.

I currently use a PIC18F8723 to do the failover. But the PIC has only 2 RS-232 ports. I can get by with only two ports because the PIC determines the packet validity on data coming in from the two sources, and uses a data selector IC to switch the input from the "good" port to the flight controller (called port 'C' above).
But the method has limitations due to the limited number of ports. It cannot switch baud rates, and it cannot deal with the new port 'D' which will be used for radar.
The radar unit will sense obstacles. If an obstacle is encountered, and the drone is in the "full AUTO" (GPS guided) mode, the drone will have to fly around the obstacle. To do so, the TEENSY will have to feed it a series of commands based on various factors, and when the obstacle disappears, it will send a series of packets to put the flight controller back into the AUTO mode.

The code in my PIC18F I'm using now is written in PicBasicPro (PBP), and even though I had to write all the ISRs myself, writing the code for that was trivial. At least for me, writing for Arduino is a LOT harder.

18

CharlesLinquist:
Yes, the data comes in either port A or B and is sent to port C. Every time that port C gets a good packet (it checks for CRCs also), it sends a short ACK (about 10 bytes) back to both ports A and B for transmission.

While I have a better picture of what you are trying to do this bit adds to my confusion.

From your earlier posts I thought the data was validated after it arrived at Port A or B and only valid data is sent via Port C

And, just to be clear, my understanding is that A, B and C represent 3 HardwareSerial ports on the same Arduino.

The process I have in mind is something like this (duplicated for Port B, of course)

check Port A for new data
copy the new data to a buffer
when all the data has arrived raise a flag

when the flag is raised
check the CRC
if the CRC is OK
copy the data to a "valid data" buffer
clear the flag so Port A can be checked for more data

if there is valid data send it via Port C

...R

19

I totally understand the logic - as I mentioned before, I have written working code for a Microchip PIC in PicBasicPro (totally interrupt-driven). But my problems are with 'C' and Arduinos.

And I'm not experienced enough with ATMEL stuff that I can easily look inside the libraries to find out what is going on. I'm trying, but struggling.

I'm learning the syntax and at the same time trying to figure out what each command does down to the byte level. For example: is Serial.read() interrupt driven, and does it fill a buffer in the background? Is there a way to have an ISR grab only one character so I can test it before deciding whether or not I want more? Is there a way to simply tell an ISR that I want 'x' characters without using a counting var? And can I put logic inside an ISR to make the decision whether or not to continue?

My dream is to find an ISR that looks for a START byte (0xFE), and when received, grabs the next byte (data length) and then loads (data length +6) bytes, set a flag and exit - all in the background.

Like I said, I can do this easily in a PIC using PBP, but I'm lost when it comes to 'C' and ATMEL. Part of my handicap is that I'm used to manipulating the processor registers and memory locations, and so much of that is 'hidden' by 'C'.

20

is Serial.read() interrupt driven, and does it fill a buffer in the background?

Yes, and yes.

The rest of it, you're over-thinking.