ISO Data logger in the field- I *think* what I want is an ad hoc network (no internet) to relay data directly to laptop

Hi,
This is my first foray into use of a wifi shield. In my case I'm using a board compatible with Wemos D1 Mini along with a datalogger module with RTC.

For now I'm just getting familiar but for that reason I want to make sure I'm on the right track intially. I have no real code yet- but I've plugged in a few examples that all seem to work as intended (to verify connections/setup). My application is I want to build a little robot who logs data from a few environmental sensors periodically and saves to the microsd as .csv or at least some format I can eventually turn into a .csv file. I would like to access that data remotely rather than have to take it apart and manually read the microSD each time. Ultimately it might be nice to figure out how to do a version that builds an overlay in realtime but that's later after I sort out the novice stuff that I'm doing now.

Anyway- so I think the first hurdle is the wifi. This will get used outside away from wifi/internet so I want to avoid relying on any internet servers/routers, and I don't care about security. I think that means this is defined as an ad hoc network (right?). I'd like to log my laptop directly to the broadcast wifi module and then read and/or transfer the data onboard. Are there any examples that capture this intent?
Thanks for any help!
-kevin

if the WiFi device has no passwords and both your robot and laptop can connect to it (assuming there is just one WiFi) then your devices just need to know about their IP addresses in order to communicate using IP/TCP connections

1 Like

If I understand right you want to have

just one ESP8266 <----------> your laptop in absence of any other WiFi-hardware (no WLAN-router etc.)

I have not tested this myself but this website seems to be promising from the title

The democode uses two ESP8266 but as one ESP8266 is acting as accesspoint it should be possible to connect your laptop too.

Another way would be to connect one ESP8266 through the USB-cable to your laptop
and and then using the ESP-NOW-protocol to communicate with a second ESP8266 modul

best regards Stefan

1 Like

gcjr: Thanks! If I can specify no passwords I will. The robot IS the wifi/D1 mini. I guess next up is learning IP/TCP. Ideally I'd just like this to emulate a sdcard but with wireless access instead of direct.

Stefan: Yep- I like that illustration that's what I want. I'm a bit surprised that this configuration seems unusual as it seems like almost the most basic wifi possible. It does sound promising- sounds like it ought to be easy enough to sub in a laptop for a second ESP8266.

there are higher level functions that spare you of that pain.

i built a model railroad throttle using an esp32 that communicates with a controller that can run on a laptop. the esp/arduino IDE has a WiFi library that allows you to specify the "ssid" and "password" to connect to a specific wifi network, as well as functions to connect to a specific application (host, port) and then to simply send/receive bytes/strings of information.

1 Like

Yes I tested the code with a short python script

Here is a modifyed version of the ESP8266-accesspoint.code

#include <ESP8266WiFi.h>
#include <WiFiUdp.h>

int packetsize = 0;

void PrintFileNameDateTime() {
  Serial.println("Code running comes from file ");
  Serial.println(__FILE__);
  Serial.print("  compiled ");
  Serial.print(__DATE__);
  Serial.print(" ");
  Serial.println(__TIME__);  
}


boolean TimePeriodIsOver (unsigned long &periodStartTime, unsigned long TimePeriod) {
  unsigned long currentMillis  = millis();  
  if ( currentMillis - periodStartTime >= TimePeriod )
  {
    periodStartTime = currentMillis; // set new expireTime
    return true;                // more time than TimePeriod) has elapsed since last time if-condition was true
  } 
  else return false;            // not expired
}

unsigned long MyTestTimer = 0;                   // variables MUST be of type unsigned long
const byte    OnBoard_LED = 2;


void BlinkHeartBeatLED(int IO_Pin, int BlinkPeriod) {
  static unsigned long MyBlinkTimer;
  pinMode(IO_Pin, OUTPUT);
  
  if ( TimePeriodIsOver(MyBlinkTimer,BlinkPeriod) ) {
    digitalWrite(IO_Pin,!digitalRead(IO_Pin) ); 
  }
}

 
// Set AP credentials
#define AP_SSID "TheOtherESP"
#define AP_PASS "flashmeifyoucan"
 
// UDP
WiFiUDP UDP;
IPAddress local_IP(192,168,4,1);
IPAddress gateway(192,168,4,1);
IPAddress subnet(255,255,255,0);
#define UDP_PORT 4210
 
// UDP Buffer
char packetBuffer[UDP_TX_PACKET_MAX_SIZE];
 
void setup() {
  Serial.begin(115200);
  Serial.println("Setup-Start");
  PrintFileNameDateTime();
 
  // Setup LED pin
  pinMode(OnBoard_LED, OUTPUT);
   
  // Setup serial port
  Serial.begin(115200);
  Serial.println();
 
  // Begin Access Point
  Serial.println("Starting access point...");
  Serial.print("WiFi.softAPConfig(");
  Serial.print(local_IP);
  Serial.print(",");
  Serial.print(gateway);
  Serial.print(",");
  Serial.print(subnet);

  if (WiFi.softAPConfig(local_IP, gateway, subnet) ) {
    Serial.println("); was successful");
  }
  else {
    Serial.println("); failed");    
  }

  
  WiFi.softAP(AP_SSID, AP_PASS);
  Serial.println(WiFi.localIP());
  Serial.print("Soft-AP IP address = ");
  Serial.println(WiFi.softAPIP());
  
  // Begin listening to UDP port
  UDP.begin(UDP_PORT);
  Serial.print("Listening on UDP port ");
  Serial.println(UDP_PORT);
}

 
void loop() {
  //BlinkHeartBeatLED(OnBoard_LED,100);

  if ( TimePeriodIsOver(MyTestTimer,2000) ) {
    Serial.print("Soft-AP IP address = ");
    Serial.println(WiFi.softAPIP());
  }  

  // Receive packet
  packetsize = UDP.parsePacket();
  UDP.read(packetBuffer, UDP_TX_PACKET_MAX_SIZE);
  
  if (packetsize){
    Serial.print("UDP-packet #");
    Serial.print(packetBuffer);
    Serial.println("#");

    if (packetBuffer[0] == 1) { 
      digitalWrite(OnBoard_LED, HIGH);
      packetBuffer[0] = 0;      
    } 
    else {
      digitalWrite(OnBoard_LED, LOW);
    }
  }         
} 

and here a small python-script that sends a "Hello World!" to the ESP8266 once per call
as it has no loop

import socket

UDP_IP = "192.168.4.1"
UDP_PORT = 4210
MESSAGE = "Hello, World!"

print ("UDP target IP:", UDP_IP)
print ("UDP target port:", UDP_PORT)
print ("message:", MESSAGE)

sock = socket.socket(socket.AF_INET, # Internet
             socket.SOCK_DGRAM) # UDP

sock.sendto(bytes(MESSAGE, "utf-8"), (UDP_IP, UDP_PORT))

and here is the serial-monitor's output

19:02:44.907 -> Soft-AP IP address = 192.168.4.1
19:02:46.902 -> Soft-AP IP address = 192.168.4.1
19:02:47.197 -> UDP-packet #Hello, World!#
19:02:48.936 -> Soft-AP IP address = 192.168.4.1
19:02:50.939 -> Soft-AP IP address = 192.168.4.1
19:02:52.904 -> Soft-AP IP address = 192.168.4.1
19:02:54.919 -> Soft-AP IP address = 192.168.4.1
19:02:56.175 -> UDP-packet #Hello, World!#
19:02:56.923 -> Soft-AP IP address = 192.168.4.1

best regards Stefan

an ad hoc network is peer to peer. a bunch of people at a meeting, communicating only with each other, no access point

therefore, no internet, no login

as opposed to a star network with an access point which requires a login, but is not necessarily connected to the internet. every device connects to the access point, which connects them to each other, which looks more like a spider than a star.

Awesome! Thanks so much! Yes, I have it working now- I can connect to it and all seems right with the world... at least till my next puzzle. On that note- one thing that's not yet clear is how I interface with it. Am I still using the arduino IDE? I'm currently connected to the device and the serial monitor displays what it should, but of course all that is coming through the USB cable and not (necessarily) over wifi. I tried accessing the IP address(es) directly and while I was able to ping them, I'm not sure if I should be using a terminal window or something else to communicate with the D1. Ultimately I'm going to want to get to a FTP or Folder type interface to reach what gets saved to the microSD.

So to summarise the functionality:
There is a device that collects sensor-data and is storing the sensordata onto sd-card.
(by the way You can do this straight-forward as comma separated values).

Then you want to remotely transfer the sensordata stored on the sd-card wireless using wifi to your computer.

Well this can be done by using UDP and a UDP-packet-receiving-python-script

But there are some more technical aspects to clear:
what will be the distance between your sensordata-collecting device and your computer?
The PCB-antenna on a Wemos D1-mini-modul is less effective than a antenna of a WLAN-router
I would expect to have this connecting working reliable over a distance of 20 meters with free sight and maybe 5 meters with some obstacles inbetween.

So would this be sufficient? Or do you expect it to work over a bigger distance?

Of course if you start optimising the pcb-antennas position you can reach higher distances
or if you use a different ESP-module which as a connector and can use an antenna with directional characteristic the distance goes up.

But then it will be easier to use 433 MHz-modules. The lower frequency compared to 2,4GHz = 2400 MHz is less weakened by obstacles.

best regards Stefan

Until know you were asking questions which is really OK.
But now it is about time to do some steps on your own.

start writing code that is your best attempt on how to do it.
It is very likely that some new questions arise. This is really OK too.

The difference will be that the questions that arise from trying to really write code will be much more specific about certain aspects.

Until now your postings can be seen as a clever way to make other people write the code for you.
I will not do this
best regards Stefan

Hey Stefan,
Sorry if I came off that way, of course I'm interested in doing the research and learning myself, and historically I've been able to achieve this by reviewing the example sketches that have been provided on forums and with the IDE and I'm continuing on that route here. I already have all I need to acheive my current milestones with what has been provided here already, I only was asking about how to interface with the peer to peer we now have. The SD part was only for context I don't expect anyone to write it for me but often if I don't provide enough information I hear criticism that I'm not relaying the whole picture. Now I have provided the big picture and catching a bit of crticism that I want someone to do the work for me- and that is not the case either. Honestly I'm quite grateful for the help I've already received and it's a turning point for me where (after years of struggle and frustration) my code is finally starting to run the way I want and the way its written is starting to make sense. This has changed my whole outlook on arduino and so my projects are quickly getting more frequent & ambitious as I move into more unknown operations. That's where we are now with wifi. So- in short: thanks for the help!

*To answer the questions of your previous post: Very short range is fine. Distance is not a concern. However, your answer of " using UDP and a UDP-packet-receiving -python-script" is probably all the direction I need and I will follow up on that myself.

If you post code - even if it is just a small sketch that does some basic stuff and then ask specific questions how to add this or that feature this shows your own effort and then everything will be fine.

best regards Stefan

I think the logical way to approach a new project (and correct me if I'm wrong) is to start with example code for the most difficult element. In this case (for me) that is the wifi element. I said when I started that I haven't found a suitable example yet and now I have. My primary hope was that someone would just offer some terms "peer to peer", "IP/TCP", "UDP-packet-receiving". It's hard to figure these things out when you don't even know the keywords. If I have other questions for this project it will surely include my own code based on this original example code that you kindly modified. I expect I will be using this wifi code often if I keep thinking up reasons for wifi- most of the things I'd make would rely on this type of connection.

peer-to-peer suggest direct communication between 2 devices -- no intermediary devices

of course it's possible, but do laptops typically communicate directly between one another or thru a Wifi "hub". in other words, i believe a packet is transmitted to the Wifi hub which then retransmit the packet to the "other" laptop connected to that same Wifi "hub"

if this were a wired coax connection using ethernet, mac addresses would allow direct communication between device. so i assume using an old Wifi hub lying around. a ssid/password may be necessary to connect to a specific wifi deive

so the learning process i went thru is first connecting to the wifi hub, then to a application on another device (ie. host, port) and then sending data to that application

when i did this with the esp32, i knew enough to create an app on my laptop which could report a connect to a device, then report received data and finally echo something back

Google RandomNerd tutorials and learn how to build a web server into the robot controller, then search for that using the networks connect system for your lap top. I have done this with an esp32 and some temperature/humidity sensors.
My next project is to transfer data from the ESP and store it as CSV on my laptop.

1 Like

Thanks, Kiwi. That answer produces a few other questions for me and still its back to trying to choose that first step/foundation for the project so that I don't have regrets after spending all this effort toward a (suboptimal?) solution. I think what you're addressing here is my question about interface- like how do I read the "hello world" through wifi rather than serial monitor/usb cable. I'll assume that the randomnerd tutorials that I'll be watching after this post will address that, but I was initially thinking I could connect with a terminal program or telnet or something to that IP address. It's still not crystal clear, but I'll figure it out shortly.

Regarding hardware: this will surely work for my application, but bluetooth would probably work equally well(I haven't yet thought of a reason I'd need significant bandwidth) with less power & maybe smaller footprint. I'd consider using an alternate microcontroller with integrated bluetooth if there was one (& RTC for that matter- would love to skip shields entirely). Supposing that was the case, is the network arrangement similar enough to use this same code in any way or am I back to the drawing board?

Please keep us posted with your ESP/CSV project- sounds like we share that goal as well!

The ESP32 microcontroller has bluetooth on board. Anyway to I did some tests with buetooth. The bluetooth library needs a lot of memory. (around 50%) so there is still room for other things to code.

An ESP32 has an RTC on board and is able to go into deep-sleep-mode and awake up again without any hardware.

If you can describe in detail what you want to do with the data when the data has been sent to your computer more suggestions can be made.

Of course there are a lot of ways how you can receive data on a computer.

You haven't specified in detail which way (bluetooth or WiFi) the data will be transferred. If you mean bluetooth you have to use the bluetooth-library and the blue-tooth functions which of course have different names than WiFi-functions.

Anyway programs should be divided into functional parts

  • a part reading in sensor-data
  • a part storing the data on SD-card
  • a part transmitting the data

There shouldn't be a lot of lines directly in "loop". The code should be well sorted into different functions.

I'm coming back on python. If you google you will find a lot of basic python-scripts that will show how to

  • send UDP-messages (which could be used to make the microcontroller start sending his SD-card-data

  • receive UDP-messages

  • write data into a textfile

  • writing python-code that creates a graphical user-interface for a more comfortable control what your microcontroller shall do

As you want to send multiple numbers aquired from multiple sensors and create a CSV-file at the end;
I recommend using the SafeString-library which makes it easy to concenate integers, floats, chars and of course strings to one big string already including the commas "," .

You just send the whole long string (even if it should be 1024 characters ) as an udp-message and a python-script stores the received UDP-message straight forward as a line into a textfile.
Each new UDP-message will be just appended at the end of the existing textfile.

best regards Stefan

1 Like