ESP32 Sending Data as UDP-Msg tiny python-UDP-listener stores data into txt-file

Hi everybody,

The original question is written below the ########### Original Question ###############

I wrote a demo-code that sends UDP-messages for data-logging purposes.
The version of this code at date 30.03.2021 is posted in post #7
The Arduino-code has a comment-section at the end that contains the python-code that does receive the UDP-messages. I tested the code with Arduino IDE 1.8.13 and Python 3.9.2 for windows.

For more info see post #7

########### Original Question ###############
I have several sensor-projects temperatures, temperaure humidity CO2 etc.
I want the ESP32's to send the data over WiFi to some kind of a central server.

I don't need fancy user-interfaces or high complex datastructures.
A simple textline that can hold up to 1024 bytes will do.
The further dataprocessing is done on a PC.

Most important for me implementing sending the data - ideally - should be simple as a single line of serial.print(myData);
something like SendToServer(myData)
with minimum overhead for configuring the connection

So what kind of "server" would you suggest?
I guess something like MQTT with all setting up the MQTT-server register MQTT-clients etc. is more work than
connect to WiFi send data (string with 1024 bytes to IP-adress of server - done.

The "server" could be a ESP32 forwarding the data over a serial interface or maybe a windows-software running on a tablet storing the data into a textfile.

best regards Stefan

Sorry, got nothing for you other than MQTT. I find that MQTT is very cookie cutter. Good luck.

The server could be a Pi acting as a LAMP server. Or an MQTT server. Or an ESP32 that pushes the data to a PC over serial. I like the LAMP solution myself, YMMV.

Hm I want to add a description of the receivers funcionality:

some kind of "server" is listing for UDP-packets send over WiFi
Whenever a UDP-packet comes in start a logfile fully automatic and store udp-message in this file as one line of text
whenever another UDP-packet rushes in add the packet one line below

Distinguishing if a UDP-packet should be added to an existing textfile or a new texfile should be created depends
on does the received a packet have a already known header or not.

So this means look up existing-logfile-headers if there is a file with the same header add textline
else create new texfile

I think any kind of "server" MQTT, SQL, XML, JSON or whatever can provide this functionality but which one does it KISS
Keep it simple stupid
new datapacket received take first 32 bytes compare bytesequence with existing files
if file found append data
or if not found create new file with the new header - done.

second thing: the files should be accessible like a harddiscdrive choose directory path double-click textfile
view data or copy file open with LibreOffice as CSV or whatever

best regards Stefan

Reads like your going to be rolling your own but if you find a thing that meets your specs let us know.

Hello,

If you have a pc that is available on the network you can use WawiLib.

After you have initialized your ethernet connection, you only need 1 line of code for each variable and you can store your data in CSV, XML, XLSX (= native excel). WawiLib is also able to create a new data file each hour, each 15 minutes etcetera.

WawiLib demo on forum

Best regards Johi.

OK I had some first rudimental success:

I found a pretty short python-code that creates a UDP-listener

# very simple and short upd-receiver found here
# https://www.studytonight.com/network-programming-in-python/working-with-udp-sockets#

import socket

sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)      # For UDP

udp_host = socket.gethostname()		        # Host IP
udp_port = 4210			                # specified port to connect

#print type(sock) ============> 'type' can be used to see type
				# of any variable ('sock' here)
sock.bind((udp_host,udp_port))

while True:
  print ("Waiting for client...")
  data,addr = sock.recvfrom(32)	        #receive data from client
  Msg = data.decode('ascii')
  print ("Received Messages: #",Msg,"# from",addr)

that receives code from a UDP-Demo-sender running on a ESP8266 nodeMCU

Though know I have to analyse and learn why the python-UDP-listener spits out all these

/x00 's

Received Messages: b'787\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'  from ('192.168.178.167', 49400)
Msg = data.decode('ascii')

does the trick

here is the ESP8266 Demo-code
This code includes some of my basic tool-functions like heartbeatblinker with onboard LED giving visual feedback code is running

on startup telling compiled-file to the serial monitor
I insert these functions into every code I write through modifying the bareminimum.ino

This also inlcudes PString for easier String-handling and non-blocking-timer-function based on millis()

#include <ESP8266WiFi.h>
#include <WiFiUdp.h>
#include <PString.h>

boolean TimePeriodIsOver (unsigned long &expireTime, unsigned long TimePeriod) {
  unsigned long currentMillis  = millis();
  if ( currentMillis - expireTime >= TimePeriod )
  {
    expireTime = currentMillis; // set new expireTime
    return true;                // more time than TimePeriod) has elapsed since last time if-condition was true
  }
  else return false;            // not expired
}

const byte OnBoard_LED = 2;
int BlinkTime = 500;

void BlinkHeartBeatLED(int IO_Pin, int BlinkPeriod) {
  static unsigned long MyBlinkTimer;
  pinMode(IO_Pin, OUTPUT);

  if ( TimePeriodIsOver(MyBlinkTimer, BlinkPeriod) ) {
    digitalWrite(IO_Pin, !digitalRead(IO_Pin) );
  }
}

unsigned long TestTimer;
unsigned long UDP_SendTimer;

int myCounter = 0;

char    UDP_Msg_AoC[16 + 1]; // always remember one extra-char for terminating zero 
PString UDP_Msg_PS(UDP_Msg_AoC, sizeof(UDP_Msg_AoC));

char *ssid = "";
char *password = "";

// receiver-IP
IPAddress remoteIP(192, 168, 178, 157);
unsigned int remotePort = 4210;  // remote port to listen on

WiFiUDP Udp;

void PrintFileNameDateTime() {
  Serial.print("Code running comes from file ");
  Serial.println(__FILE__);
  Serial.print(" compiled ");
  Serial.print(__DATE__);
  Serial.println(__TIME__);
}

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.println("Setup-Start");
  PrintFileNameDateTime();
  Serial.print("Connecting to "); 
  Serial.println(ssid);

  WiFi.persistent(false);
  WiFi.mode(WIFI_STA);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    BlinkHeartBeatLED(OnBoard_LED, 200);
    delay(200);
    Serial.print(".");
  }
  Serial.print("\n connected.");
  Serial.println(WiFi.localIP() );
}

void loop() {
  BlinkHeartBeatLED(OnBoard_LED, BlinkTime);

  if (TimePeriodIsOver(UDP_SendTimer, 2000) ) {
    
    UDP_Msg_PS = myCounter++; 

    Serial.print("Send UDP_Msg #");
    Serial.print(UDP_Msg_PS);
    Serial.println("#");

    Udp.beginPacket(remoteIP, remotePort);
    Udp.write(UDP_Msg_PS, sizeof(UDP_Msg_PS) );
    Udp.endPacket();
  }    
}

@JOHI do you have a ready to run demo-code for an ESP8266 /ESP32 just sending a counting up number and a configuration-file that I just import into WaWiLib that writes the UDP-received data into a textfile?
Without such an working example WaWiLib seems to need some hours of reading the manual and adapting code.
best regards Stefan

Hello,

So I made some progress and have a proof of concept working here.
The ESP32-code connects to the local WiFi synchronises time with internet-time and sends UDP-messages with a header a timestamp and userdata. The header is used in the receiving python-code as filename.
This means if you have different headers - for each header - a own txt-file is created. Each UDP-message is added at the end of the TXT-file with a timestamp.

So this is a very very simple but quite effective datalogging functionality to collect data from multiple ESP32 devices as CSV-files with timestamps.

If you test or use this code drop a short private message as a "thank you"

as the code exceeds the 9000-characters limit I attach it as *.ino-file

best regards Stefan

P.S.: If some body wants to ask why didn't you reduced the code to be shorter?
Because my personal style is to increase learning through additional comments and additional functions rather than increase learning through analysing uncommented code "the hard way"

Test-ESP32-UDP-Sender-with-NTP-TimeStamp.ino (9.33 KB)

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.