I am working on a large LED installation project for a local library. Project description is here.
I followed, basically, the approach of transmitting the LED info over websockets presented in this project How to Build a Massive LED Wall on a Budget. There are 5 ESP32 controlling segments of the wall. Here is the code for the receivers.. I'm sending data over websockets from a nodeJS app, which is just an array of 8 bit RGB values.
I'm getting an issue where some of the ESPs lag behind others, and generally the animation is jerky and stuttered. It looks like the animation gets behind, but occasionally 'catches up' to the rest of the panels. It always seems to be the same panels. The third is a little slow, but the fourth is a real pain. Here are some videos of the issue:
I'm not sure where the lag is occuring. The NodeJS app is sending data to each panel in a loop over websockets, so I know all the video data is being sent at the same time. I'm guessing those ESPs are running too slowly to keep up with the websocket data and it is buffering on the controller. Does that sound right? I was looking into some way to keep all the boards in sync using a clock from the server, but I didn't want to jump down that rabbit hole until I understood the problem better.
OR, another option I've been contemplating is having a small computer physically connected to the ESP32s to receive and send data, but there are 11 total, (6 for sensors). Is there a way to connect that many ESP32s to one computer?
BTW, each ESP32 is getting good 5V power, there don't seem to be any dips in voltage. The wifi connection is consistent as well.
OK, so I think ESP Now has a data transmission limit of 250 bytes? Each receiver is receiving 1952 bytes of data over 5 receivers, so even if I spread that out over multiple transmissions, i don't think the speed is there for motion graphics. This is probably why I forgot about it.
Hardwiring the ESP32's through serial USB has revealed that the ESP32's can simply not keep up with the data in general. Looping an array of 1952 ints as fast as possible, I can get up to around 28 frames per second. Not bad, but that is pushing it as fast as it can go.
Looking into it, ESP32s have a clock speed of 240Mhz. So I ordered some Teensy's which run at 600Mhz and I was able to the the fps up to about 50. Of course, Teensy's don't have Wifi builtin.
So now I wonder if a combination of pushing the ESPs to the max along with potential packet loss is causing issues. I'm not sure why it is specifically the 3rd and 4th controller, I am going in later today to do some testing.
Measuring the output of the ESP32's using websockets with my logic analyzer, I'm seeing a bunch of lag and catchup situations. In the video below, things are going ok, then a random chugging of data output, then it smooths out again. This happens a lot, and I think might be part of the problem.
Running the data over serial results in a much cleaner pattern.
I'm wondering if there is a possibility to buffer some frames to compensate for this. I'm also wondering if I should use more microcontrollers to reduce the size of the arrays each one has to handle. I'm just not feeling good about the number of controllers i'm using, which may be an indication I should be using more powerful controllers, like a Raspberry Pi.
So, I can:
lower the load on the ESP32s by adding more controllers. Though, this doesn't manage the websocket lag
1a) Adding a frame buffer on the ESP32, if there is enough memory for that
Get a more powerful controller, like a Raspberry Pi or two
hardwire everything, which introduces a new set of hurdles (specifically, the graphics are powered by p5.js, so it has to run in a browser. The computer would not be directly accessible, so some sort of remote control would be necessary).
Only a generic one that using IP for realtime data is problematic because there's no guarantee of delivery or of what the latency will be or even that the packets will arrive in the order they were sent. My background is telephony, VoIP takes care of these problems by using UDP (there's no point using TCP, if a packet is lost it's too late to request it again) and using a small (buffers add delay, you don't need much delay on a phone call for it to be noticeable) buffer.
OK, yeah, I'm getting the feeling that Wifi just isn't reliable enough for live streaming to the ESP32s. I think I'm going to try to re-make this with hardwired connections.
