Use an ESP8266 to redirect an URL to a local IP / Captive portal?

Hi everybody,
I'm new to this forum and a little less new to the Arduino world. Over the past months I've used a couple of Arduino and ESP8266 (Wemos D1 mini) boards for homebrew sensors for my home automation (HA) systems. Even if I managed to create some own sketches, I'm still far away from being an expert or a programmer.
Thus my first question I wasn't able to solve by myself:
I have a weather station (Ventus W830) that sends its data via the local Wifi to wonderground.com. There's no regular way to grab the data locally and send it to the HA system.
But since a while there is a module for my HA system (fhem) that is able to process data coming in in the Wonderground format. To get the data there, it's necessary to redirect rtupdate.wonderground.com to the local IP of the HA system, e.g. 192.168.178.100/fhem (and it needs to be directed to port 80, hard coded in the weather station).
Of course, I could achieve this with dnsmasq or pi-hole and iptables on a Raspberry, but I don't want tu set up a Pi only for this purpose, nor do I want to run all my DNS traffic through the Pi.
So I started looking for a solution with a Wemos D1 mini lite I had lying around.
I've tried quiet a few things, especially the Captive Portal examples , but no success. The only thing I achieved was to redirect all URLs to the web server running on the Wemos itself.
As far as I understand, the Wemos needs to act as a softAP that connects itself to the "real" wifi the HA system is connected to, and then use a redirect, but I just can't figure out how to do this.
Any help to get me on the right way is greatly appreciated.
Thanks & best regards
Martin

can't you read the data back from the cloud?

Yes, that's how it is working now. But the Wonderground API will be closed (https://apicommunity.wunderground.com/weatherapi/topics/end-of-service-for-the-weather-underground-api), and it wasn't very reliable either. For this reason, I'd prefer to have my data locally :wink:

if redirected the data to the esp8266, you can resend it to HA.
something like this

Thanks Juraj, but I'm afraid that understanding what is said there still is beyond my horizon. As far as I understand, they are talking about bridging two ESPs in order to get a wider wifi net? How could I "translate" this to my issue?

dadoc:
Thanks Juraj, but I'm afraid that understanding what is said there still is beyond my horizon. As far as I understand, they are talking about bridging two ESPs in order to get a wider wifi net? How could I "translate" this to my issue?

you receive the data in esp8266 and send them to HA.

I've tried to compile this:

#include <ESP8266WiFi.h>
#include <ESP8266WebServer.h>

const char* ssid = "SSID_Internet";
const char* password = "qaz12345";

const char* newssid = "SSID_Extender";
const char* newpassword = "qaz12345";

void setup(void){
  Serial.begin(115200);
  Serial.println("");

  // set both access point and station
  WiFi.mode(WIFI_AP_STA);
  WiFi.softAP(newssid, newpassword);

  Serial.print(newssid);
  Serial.print(" server ip: ");
  Serial.println(WiFi.softAPIP());

  WiFi.begin(ssid, password);
  int wifi_it = 0;
  int wifi_status;
  while ((wifi_status = WiFi.status()) != WL_CONNECTED) {
    Serial.println("Trying to connect to wifi");

    wifi_it++;
    delay(500);
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  long rssi = WiFi.RSSI();
  Serial.print("Signal strength: ");
  Serial.print(rssi);
  Serial.println("dBm");
}
void loop() {

  if (!serverClient || !serverClient.connected()) {
    serverClient = server.available();
  }
  if (!serverClient || !serverClient.connected()) {
    if (client.connected()) {
      client.stop();
    }
    return;
  }

  if (!client || !client.connected()) {
    if (!client.connect(raspiIP, MQTT_PORT)) {
      Serial.println("Can't connect to MQTT");
      delay(1000);
      return;
    }
  }

  while (serverClient.available() > 0) {
    client.write(serverClient.read());
  }
  while (client.available() > 0) {
    serverClient.write(client.read());
  }
}

But it seems that this is not the right way, as I get:

In function 'void loop()':

weiterleiter:44:8: error: 'serverClient' was not declared in this scope

   if (!serverClient || !serverClient.connected()) {

        ^

weiterleiter:45:20: error: 'server' was not declared in this scope

     serverClient = server.available();

                    ^

weiterleiter:47:8: error: 'serverClient' was not declared in this scope

   if (!serverClient || !serverClient.connected()) {

        ^

weiterleiter:48:9: error: 'client' was not declared in this scope

     if (client.connected()) {

         ^

weiterleiter:54:8: error: 'client' was not declared in this scope

   if (!client || !client.connected()) {

        ^

weiterleiter:55:25: error: 'raspiIP' was not declared in this scope

     if (!client.connect(raspiIP, MQTT_PORT)) {

                         ^

weiterleiter:55:34: error: 'MQTT_PORT' was not declared in this scope

     if (!client.connect(raspiIP, MQTT_PORT)) {

                                  ^

weiterleiter:62:10: error: 'serverClient' was not declared in this scope

   while (serverClient.available() > 0) {

          ^

weiterleiter:63:5: error: 'client' was not declared in this scope

     client.write(serverClient.read());

     ^

weiterleiter:65:10: error: 'client' was not declared in this scope

   while (client.available() > 0) {

          ^

weiterleiter:66:5: error: 'serverClient' was not declared in this scope

     serverClient.write(client.read());

use WiFiServer, not WebServer

Do you mean

#include <ESP8266WiFi.h>
#include <WiFiServer.h>

?
Same errors. Taking the stuff from
C:\Users\xyz\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.2\libraries\ESP8266WiFi\src
after downgrading from 2.5 because I read that 2.5 might lead to errors.

try the basic examples for WiFiServet and WiFiClient

Hi,
I‘m still not sure to which WifiServer examples you are . At Arduino/libraries/ESP8266WiFi/examples at master · esp8266/Arduino · GitHub there is a WifiHTTPS sample, but I don‘t believe that this is what you mean?

WiFiManualWebServer

Thanks Juraj! But I still don‘t understand how the server should „know“ that it must listen to http://rtupdate.wunderground.com? Theres no way to setup the Esp‘s IP in the weather station as target.

dadoc:
Thanks Juraj! But I still don‘t understand how the server should „know“ that it must listen to http://rtupdate.wunderground.com? Theres no way to setup the Esp‘s IP in the weather station as target.

I thought you already solved this with capture and can read the data sent to wunderground on esp8266.
I never tested a capture portal

The capture portal redirects any incoming URL to the web server running on the ESP. But what I didn‘ t achieve is to redirect the incoming URL to the HA Raspberry‘s IP instead of processing it on the ESP‘s web server. Should be something like dnsmasq or pi-hole, but I wasn‘t sure such a thing exists for the ESP.

dadoc:
The capture portal redirects any incoming URL to the web server running on the ESP. But what I didn‘ t achieve is to redirect the incoming URL to the HA Raspberry‘s IP instead of processing it on the ESP‘s web server. Should be something like dnsmasq or pi-hole, but I wasn‘t sure such a thing exists for the ESP.

you can do the redirect only in the AP of the esp8266. the HA can't be connected to this AP, it must be connected to your (W)LAN. but the esp8266 can do AP and be connected to you WLAN. so it can only redirect to itself and then resend the recived data to HA

must be connected to your (W)LAN. but the esp8266 can do AP and be connected to you WLAN. so it can only redirect to itself and then resend the recived data to HA

I completely agree with you. I have https://github.com/esp8266/Arduino/blob/master/libraries/DNSServer/examples/CaptivePortalAdvanced/CaptivePortalAdvanced.
running, and it does the first part of what you are saying (i.e. works as AP and connects to my WLAN. But I still have no clue on how to send the data packages (?) the weather station tries to upload to wunderground to the HA. I‘m already afraid that it might be something related to bitwise shifting in telegrams etc.?

did you try to print what is received?

Not yet, because the weather station is 1.500 kms away (holiday home in Spain). I‘ll have to think about how to create a simulator here.
Afaik the PWS protocol is used: Weather Feedback

Just came across a station project for the ESP. Seems they made a wunderground library, ESP8266-Color-Weather-Station-v8/WU library at master · fowlerk/ESP8266-Color-Weather-Station-v8 · GitHub
But as far as I understand it‘s only for a client scenario.