One ISR, four serial ports?

Hi

I want to build an Artnet to DMX node based around Toni Merinos code.

According to his notes, Toni's code won't support 4 DMX outputs as the CPU isn't fast enough. I believe this is because of the overhead of running 4 serial port interrupt service routines at once. Would it be possible to tie all of the serial ports external clock pins together and just have one master serial port and one ISR for all four serial ports? If not then I will look in to bit banging the DMX outputs.

Thanks

Can I use one ISR to transmit data over all four serial ports of an Arduino Mega?

I want to transmit 4 universes of DMX data, one universe per serial port. This is for an Artnet input/DMX output node. Each serial port will be running at the same speed so if I set up one ISR to be triggered when serial port 0 has sent its data can I assume that all of the serial ports have sent their data, and therefore are now ready to send new data? Doing this in one ISR will be more efficient that having four separate ISRs running, won't it?

Gyro_Gearloose:
Can I use one ISR to transmit data over all four serial ports of an Arduino Mega?

You shouldn't try to use an interrupt to send data over any serial port. Sending serial requires interrupts that are turned off during your ISR. If the buffer just happens to be close to full when you send, then you can deadlock your code. Let the ISR set a flag to let the main program loop know that it is time to send data and let the main program loop do the sending.

A username like that suggests you are a person of a certain age :slight_smile:

I preferred Foghorn Leghorn myself.

...R

Robin2:
Foghorn Leghorn myself.

I say I say son, you're giving yourself away there.

@Gyro_Gearloose, please do not cross-post. Threads merged.

Delta_G:
You shouldn't try to use an interrupt to send data over any serial port.

I was thinking of using the 'TX complete' interrupt of serial port 0 to trigger sending the next byte of data to each serial port. I'm guessing that using one ISR to service all four serial ports will be more efficient than four separate ISRs or four separate 'if (serial port n TX complete)' statements.

The problem is can I assume that all serial ports will be in sync? Is there anything I can do to force them to sync on initialisation, such as sending a zero to all four serial ports and then waiting for all of them to signal TX complete? Do I even need to do that? If I had access to a scope I'd have a look at all four TX pins to see if the serial data frames line up or not.

Sorry about that. Can you rename this thread to ‘One ISR, four serial ports?’ please? It makes a bit more sense given the direction this thread is going.

Thanks

They won't necessarily all be in sync. The data is going out via a set of interrupts and those won't all start at exactly the same time. You really have almost no control over it even being close as there are other interrupts which might be handled in between.

Why do you need to know the instant the data is out? The Serial line is buffered. You dont have to wait for one transmission to finish to load the data for the next one.

I think you're getting confused between the serial port hardware and the serial library. The hardware does not use buffers or interrupts to send individual bytes, only the software library uses these and I won't be using the existing serial library.

The reason I want to use one interrupt to send the data is that the DMX data needs to be sent continuously at 250kpbs. I need to know when one byte is sent so I can immediately send the next one. Since the hardware doesn't use buffers I'll need to use interrupts :wink:

There is a way to get the serial ports to work in sync by tying their external clock pins together (XCK0 to XCK3),and setting one port to be master and the others to be slaves. Unfortunately this is not possible on any Arduino as those pins are not accessible via the headers.

What I think will work is if I write the data to each serial ports UDR register starting with port 0, and then wait for port 3 to finish. Since Port 3 was given its data last, it ought to be reasonable to assume that since it has finished transmitting all of the other ports will have finished transmitting as well.

Gyro_Gearloose:
I think you're getting confused between the serial port hardware and the serial library. The hardware does not use buffers or interrupts to send individual bytes, only the software library uses these and I won't be using the existing serial library.

The reason I want to use one interrupt to send the data is that the DMX data needs to be sent continuously at 250kpbs. I need to know when one byte is sent so I can immediately send the next one. Since the hardware doesn't use buffers I'll need to use interrupts :wink:

I know what you are planning to do. And I'm pretty sure the HardwareSerial library would let you keep the software buffer full and keep the serial lines constantly sending. You are just reinventing the wheel. Why?

Is it critical to anything else that the signals be in sync? Or is this just an exercise to try to save the interrupt resources?

I don't know a lot about mucking around with the USART at the hardware level. So I'll leave you to someone else with that.

Delta_G:
And I'm pretty sure the HardwareSerial library would let you keep the software buffer full and keep the serial lines constantly sending.

Assuming there are bytes A,B,C to be sent over Serial and bytes A1, B1, and C1 to be sent over Serial1 I wonder if it is essential to know that byte A for Serial is sent at the same time as byte A1 for Serial1 rather than sending (say) A and B1 at the same time.

Gyro_Gearloose:
Since Port 3 was given its data last, it ought to be reasonable to assume that since it has finished transmitting all of the other ports will have finished transmitting as well.

That sounds reasonable to me and worth testing.

...R

Delta_G:
Is it critical to anything else that the signals be in sync? Or is this just an exercise to try to save the interrupt resources?

The signals don't need to be in sync, but the AtMega2560 apparently can't have four serial port ISRs running, and receive and parse network data at the same time.

I have been looking at Toni Merinos Artnet DMX node, and from what is written in the comments, running four serial outputs with four separate ISRs is not reliable at all. I believe this is because of the overhead of entering/leaving the interrupts, along code duplication within the ISRs, is taking so long that some interrupts get missed.

Gyro_Gearloose:
The signals don’t need to be in sync, but the AtMega2560 apparently can’t have four serial port ISRs running, and receive and parse network data at the same time.

I have been looking at Toni Merinos Artnet DMX node, and from what is written in the comments, running four serial outputs with four separate ISRs is not reliable at all. I believe this is because of the overhead of entering/leaving the interrupts, along code duplication within the ISRs, is taking so long that some interrupts get missed.

You are making a lot of un-substantiated statements there which may or may not be correct.

Is that guy’s code simple enough that one could make sense of it in a few minutes? If so, can you post the code?

Also, can you explain Artnet and DMX in a few sentences so I don’t have to learn a whole lot of stuff that I have no interest in?

…R

Theres links to resources on DMX and Artnet, and Toni's code at the top of this thread, but in a nutshell DMX is a unidirectional communication protocol for controlling theatre/stage lighting equipment. One DMX data stream is made up of up to 513 bytes sent over an RS485 bus at 250kbps. This is known as one 'universe' of data. Once the data has been sent there is a short break where the TX port has to be held low for 92 microseconds, and then the data is sent again even if it has not changed.

Artnet in its simplest form is a way of sending multiple 'universes' over Cat5 Ethernet networks, and for lighting equipment to send information back to the controller. It uses a simple UDP packet structure for moving data across the network.

As for Toni's code, I'm starting to think that I should just write my own. It uses a third-party DMX library which uses a mix of compiler directives and 'if...' statements to select which serial ports to use. It also allows DMX reception via the serial ports, which is something I won't need.

I have had a quick look at the V0.2 code in that Toni Merrino link. It is much too complex to get any sense of how it works or what it does in 5 or 10 minutes.

Where does Artnet come into it if you are just trying to send data over 4 serial ports? Is the data coming in via Ethernet and being sent on via Serial?

Could you feed the same Ethernet data into two Arduinos and have each one send out two of the 4 data streams?

...R

What I'm trying to do is create a node which takes in Artnet data from some lighting control software, and translates it into DMX which I can send to lighting equipment. Commercial units cost many hundreds of Pounds/Dollars/Euros, so I was hoping to make something a bit cheaper :slight_smile:

The bottleneck is unlikely to be sending the data. However the naive way of sending that data over serial will block rather badly. If you have a 513-byte buffer and you try to send that whole buffer, only the first 64 bytes will fit into the outgoing buffer which is emptied by interrupts. The code will block until all 513 bytes have been put into the buffer (and 449 bytes have been sent.)

If you increase the size of the outgoing serial buffer to larger than 513 then you can dump the entire universe into the outgoing buffer and the interrupts take it from there. The main program doesn't need to spend any attention on that.

The Arduino core is written in a way that should allow you to override the default serial buffer sizes with a simple #define. I've never had any success with that so I have edited the core files on my system to suit the main things I'm working with (80-byte NMEA strings.)

The secret sauce is the Serial.availableForWrite() function. You can use this to ensure that you never try to write more into the Serial buffer than it has space for. So your program never blocks waiting for the buffer to empty. Even if you stay with the default buffer sizes, your own program can write smaller blocks and always try to keep the 64-byte buffers from going empty.

Once you are sure that you're not waiting for 449 bytes to be transmitted, then you can work on any other performance problems in the program. Analyze how much time it spends in each part of the code and find the slowest one. Try to make that faster. Split it into smaller units that take less time on their own, giving other processes (like refilling the outgoing buffers) more opportunities to do their work.

If you need more memory for the big buffers or more raw processor speed, consider upgrading to a Teensy 3.6, which has 6 hardware serials as well as much faster 32-bit processing.

MorganS:
The bottleneck is unlikely to be sending the data. However the naive way of sending that data over serial will block rather badly. If you have a 513-byte buffer and you try to send that whole buffer, only the first 64 bytes will fit into the outgoing buffer which is emptied by interrupts. The code will block until all 513 bytes have been put into the buffer (and 449 bytes have been sent.)

If I understand it the whole purpose of this Thread is to avoid that - by only sending byte 2 after byte 1 is known to have been sent based on the interrupt from the USART.

...R

Robin2:
If I understand it the whole purpose of this Thread is to avoid that - by only sending byte 2 after byte 1 is known to have been sent based on the interrupt from the USART.

...R

This is exactly what the Arduino serial library does anyway.

I think there is a tendency to think that because we've sent data out using calls to the serial library, the cpu doesn't need to do anything further because the serial port hardware will take care of sending the data.

Unfortunately this is not how the hardware works. All the hardware does is send data one byte at a time and then tell us when it has sent it. It is then up to the software to send more data.

The reason I started this thread was because I'd found some code which seemed to almost do what I needed (receive Artnet data and send multiple streams of DMX data out again). However, comments in the code by the author said that using 3 or more serial ports at once to send the DMX data didn't work as expected. Apparently, having three or four serial ports running at once resulted in the DMX data not being sent out fast enough. This, to me, sounded like the interrupt routines involved in controlling the serial ports were getting in each others way. Due to the way interrupts work, when one interrupt routine is running, other interrupts will be ignored until the current interrupt routine has finished. Obviously this will cause problems if we have three or four interrupt routines running at once.

It is my hope that by combining four interrupt routines in to one I will avoid this problem. If not, then I'll just have to dig out my old Parallax Propeller dev. board. 8 cores at 80MHz will give me a bit more processing power to play with :smiley: