I am planning on using some form of communication to send logs from multiple Arduinos to a central server location, parse through those logs, and set up alerts if needed. In terms of materials, I am using an Arduino Mega 2560 with Ethernet shield.
This is pretty much a theoretical question because I haven't implemented anything yet, but I'm having to decide between using syslog or TCP for this. My main concern is about the resource needs of the syslog library – specifically any additional flash and RAM needed. I can't find much on this.
The program I would be running already takes up about this much memory:
Sketch uses 50664 bytes (19%) of program storage space. Maximum is 253952 bytes.
Global variables use 2582 bytes (31%) of dynamic memory, leaving 5610 bytes for local variables. Maximum is 8192 bytes.
Does anyone have thoughts or input on this? Thank you.
never used syslog so cannot comment
there are many examples of TCP and UDP client/server applications implemented in many languages on the internet, e.g. for a start do web search for arduino ethernet tcp
what language would you be looking at for the server?
would you store information in a database for processing?
would you wish to plot graphs, setup triggers when limits are reached, etc?
How many messages per second and how many bytes per message?
The main problem with libraries is trying to figure out if they are any good or not. Becoming dependent on a library that looked promising but turns out to be a buggy mess can turn your project into a living hell.
The Arduino de-facto standard for moving data around is MQTT which uses TCP as the transport layer. It doesn't know anything about data formats, it just moves chunks of data. The Arduino MQTT libraries are mature, reasonably fast (10s of messages per second) and there are lots of examples on the web.
Personally I would format the data as a Syslog packet, or your own custom format and send it over MQTT (TCP) and write my own MQTT to Syslog converter to run on the server. Obviously this is more work than just using a single Arduino library, however it's a low risk solution as the only bugs in the system will be your bugs and therefore fixable by yourself.
I see! Thank you for the specific search term; I will look for examples using TCP. I already have some functions using Udp over ethernet (sending back variable values for display, setting EEPROM values from a desktop application), so would I be able to use TCP in addition to Udp? Sorry if this is an obvious question. I am not experienced with this.
Sorry, I am not sure what exactly is meant by server language, but it will be a Linux server. The information will be stored and picked up by Splunk for visualization and alerts (for example, if a specified current limit is reached).
As a rough estimate, maybe 80 characters every 10 seconds.
Thank you for explaining that - I had seen MQTT around but didn't know exactly what it was. I'd like to try out the library as well just to see how it works and all, but this is great.
you can run multiple TCP and UDP clients and servers on a processor (e.g. ESP32) assuming it has sufficent CPU power and RAM and depending on traffic etc
when I asked about server language it was find out what language you were going to use if you implemented your own server code, e.g. C++, Java, C#, Python, etc. It sounds like you are planning to use an existing server application.
if the server supports it MQTT is worth considering as micros have libraries to support it, e.g. try a web search for esp32 mqtt and arduino ethernet mqtt
I use MQTT on ESP32/ESP8266 devices to transmit data to the MyDevices/cayenne desktop for visulation