How to send live data to a local server?

I'm using an Arduino Uno to sample data from an MPU6050 and flex sensors at about 50 samples per second. I want to be able to send live data to a local web server which will then handle that data and send it to a Jetson Nano. I must use the Uno (AVR architecture) since I'm using the Jeff Rowberg library for the MPU which has crashing issues and is only fixed using a timeout function that is implemented in the wire.h library for AVR boards. To enable Wi-Fi communication on the Uno, I have ordered an esp8266 Wi-Fi driver. What is the best method is which I can send this data wirelessly to a local web server? I have been looking into http requests and websockets, however I'm unsure with what's compatible with the current hardware I'm working with and what is the most reliable for handling live rapid events. Any form of suggestions will be helpful!

Why do you need a web server?

We are creating a web app that will handle the input data and delegate that towards the receiving side (Jetson Nano) which will be controlling a robotic arm. Also, there will be other commands being sent via the web app and a live video feed. My question at the moment is handling the input data side of things.

I guess my question was Why bother with http ans not use simple sockets?

I haven't decided on anything yet.. this is why I'm here to seek help. I mentioned looking into both http and web sockets and need to make a decision on what's best. This includes being sure on the compatibility with the current hardware I'm using.. i.e the Arduino Uno and the esp-01 8266 wifi driver.

That doesn't sound like a convincing reason for using a Uno - and maybe not the Jeff Rowberg library either. Are you quite sure that you cannot use an ESP8266 device and retire the Uno?
Fifty samples a second is not that fast but you might find the ESP's extra speed and memory a godsend.

We're controlling a robotic arm using a wearable glove. I found Jeff Rowberg's library very powerful in accomplishing what we need as it gives relative tilt (yaw pitch roll) on 3 axis without any kind of drift. I have tested my code locally and the robotic arm is able to mimic movement really well on 3 motors. I'm not sure about replacing the Uno with the esp8266 board but I can look into it.. my only fear is that the MPU freeze issue reoccurring. It took me 10-15h looking through forums and the library to fix the problem on the Uno and I'm very hesitant changing the hardware I'm currently using. I also tried a nano 33 iot and it also gave me the freeze problem with no fix on that architecture that I can currently find.

That was what triggered my question. Web sockets’ handshake happens over http too, it was designed for the web (although is not HTTP)

There is nothing fundamentally wrong in using http but it adds a layer that seems unnecessary if you only want to send a payload from one MCU to another.

Also if you hope the UNO will have enough memory and speed to handle that communication as well as a video feed, you are up for a big disappointment.

I would get a ARM / ESP32 based arduino for the positioning system ( The arduino Nano 33 BLE Sense come with an on board IMU for example ) and a separate ESP32 or another gig (RPi) for the video streaming piece.

do you have a working WiFi networking from Uno over esp8266?

I have 13 ESP32's sending and receiving data throughout my house using MQTT.

The ESP32's send and receive data to/from a Raspberry Pi that does the MQTT thing, the FTP server thing, the image processing thing, and a few other things.

I'm able to go to my website, see the house temps and set AC on or off or higher or lower or switch to heat, or change the humidity, or water the plants or a few other things.

Oh the RPi, takes all the ESP32's data and inserts/updates my websites db.

There could be something for you here.
https://github.com/Links2004/arduinoWebSockets/tree/master/examples/esp8266/WebSocketServer_LEDcontrol
https://tttapa.github.io/ESP8266/Chap14%20-%20WebSocket.html

As I mentioned in one of my earlier comments, the receiving side will be a powerful Jetson Nano. The robotic arm and video stream will be connected there. The Uno is only used to sample the input data and send it over to the server. Do you think I would be able to just parse the incoming serial data from a live log.txt file since the Uno will be connected to a laptop anyways? That way I can scrap the idea of needing a wifi enabled board.

Not yet, I received the esp-01 part yesterday. Excuse my ignorance, maybe I'm wrong about how the esp modules work but at the end of the day, my goal is to send the sampled data over to the local server where my other partners can then handle that data and do as they wish with it. Whether I use a wifi enabled board or somehow parse the serial data coming to the laptop.. it doesn't matter to me as long as that data is sent somehow to the local server.

You betcha:

Serial output directly into NodeRED - Arduino for STM32 (stm32duino.com)

This looks great and might actually work. I'll be sure to bring up the idea with my team members. Do you think I can reach out to you personally if possible?

Most here do not participate in extended off-forum PM *; I however do but try to limit them as it is difficult to collaborate across all world-wide timezones. * see note

NodeRED is multiplatform, a commercial grade enterprise solution with great documentation: Documentation : Node-RED (nodered.org)

  • Note Added:
    Also, off-forum PM sometimes reveals tricks and techniques that would be beneficial to the forum as a whole.

Thank you for the NodeRED suggestion, I've been testing it out the past couple hours and it seems like a very powerful tool. In your experience, you think we would be able to then have a complete local setup with the following MPU/sensors --> Arduino Uno --> Serial --> Laptop --> NodeRED --> server/webapp <--> Jetson Nano --> robotic arm? That is really the whole premise we are trying to achieve.

Yes, that will work with a side-benefit that the web-server is an installed component of NodeRED. My reluctance in stating unequivocally this is ideal is based on too many unknowns beyond NodeRED: for example the server-webapp will introduce a delta-T as will manipulation/message and protocol conversion in the Jetson Nano and software rules to drive the arm.

Thus, at NodeRED, the serial connector is only bound by the maximum USB serial link. NodeRED can parse and manipulate the JSON stream, do bounding rules, and math. The webserver at this point is essentially a summary view of anything coming downstream. My understanding is this will migrate upstream to the Jetson Nano for AI.

These kinds of "systems" are difficult to characterize; almost always empirically gathered data necessary for qualification.

Ray

Regarding the Jetson Nano, it is meant to receiving incoming data from the server and serve those values directly to the robotic arm. I'd like to mention that there is a PCA9865 driver with 6V supply that is connected as well. This allows the 4-6 servo motors on the arm to be easily controlled using pwm. The reasoning behind the Jetson Nano is that we plan on having object dimension recognition running (opencv) which is to be live streamed back to the server, in order for the user to view this live feed on the web app. In all honesty, the video stream could be somewhat treated as a separate entity and we could add some wireless Arduino to service the robotic arm. We're open to suggestions if you think there might be a better design of the overall system.

I second the opinion that Arduino Serial ➜ PC is an option, you could communicate at 1Mbauds for example to ensure there is limited latency due to that communication

What you have on the PC can be anything - a Python script is pretty effective too.