Arduino webclient unable to reach webserver on internal network

Hello,

I am new to Arduino and trying out some of the examples. I have a MXChip IoT DevKit board (AZ6133).
I was trying out the sample code "WebClient" with some web sites and some internal web server addresses (my NAS). While the web client works OK in call web sites on the internet, it cannot reach the internal ones however.
My conclusion: the sample code is fine, as it is able to setup Wifi and make a Http call (and receiving reply). There must be something wrong in the network then?

I first checked my router (Ziggo ConnectBox) - it appears that my board has been assigned an internal IP.
I can also ping from my router to the Arduino board, but I cannot ping from my NAS (on the same internal network) to Arduino. When I try a traceroute, it only shows time-outs however.
When I setup a Web Server (using the sample) on my Arduino, I get the same behaviour, but I can reach the Arduino from my local PC (Mac). Still NAS is not able to ping to Arduino.

Could it be that the Arduino is going straight to internet, without being able to re-enter my internal network (which seems logical, but why would it go to the internet anyway, if I am trying to reach a local network IP)?
Any help is appreciated! Thank you in advance,

Maurice

You didn't provide enough information about your network to track the problem down.

How is the NAS connected? By Ethernet or via WiFi? If you connect a notebook to the WiFi can you connect to the NAS from there?

Your router seems to be an OEM rebranding of your internet provider. Can you provide a link to it's manual in English?

Hello Pylon,

Thank you for your quick reply! Sorry for missing some important info in my previous post; you’re right!
My router is endeed an OEM, and it turns out to be an Arris. I found the English manual here:

My nas (Synology DS415+) is connected via Ethernet wire, like my pc desktop.
I can reach my NAS using a laptop over WiFi though, there is no problem I reaching the NAS in my home network; I use it for various server functions (e.g. plex on Apple TV, also wired) and I use both phones and iPad over WiFi to connect the NAS etc. I also tried to reach other devices in my home network (both wired and wireless) from the arduino webclient (hue bridge, belkin WiFi repeater etc), but I could not reach any of these from my arduino web client.
So in my view, the Arduino traffic is stopped at my router, but then again it is strange that I can reach web sites on the internet...

Does that mean you tried to access your NAS by IP? If you tried to access it by name it probable that you use a functionality (mDNS) not available on the Arduino by default.

What have you set your IP details to be?
If fixed, what is the gateway and the dns server set to ?

Hello!

@Pylon: no, I connect my NAS via IP (192.168.178.144). I also tried by name, but then the Arduino cannot resolve the name (for my dekstop PC this works OK, though)

@leibholt: my NAS IP is fixed (192.168.178.144), assigned by the router/gateway (192.168.178.1) and the IP of the Arduino as well: 192.168.178.200. So all are in the same subnet (192.168.178) and my NAS is able to ping all devices in this subnet, except for the Arduino. NAS can also connect both to wired and wireless devices like my ipad and iphone. Furthermore, when I scan my network using the mobile app Fing (https://www.fing.com/), all devices are immediately visible, including the Arduino.

3 Step analysis approach:
a) Please post your adopted webclient code how you tried to reach the NAS.
b) What's running on the Synology (beside DSM) what should be reached by the Arduino?
c) Please show us a snapshot of your browser accessing this resource on the Synology including the addressbar of your browser (the same you tried with a) )

Hi Noiasca, thank you for your reply! Please find my answers to your questions below:

a) Please post your adopted webclient code how you tried to reach the NAS.
I am using a standard example included with my board (MXchip/AZ6133) - the MQTT client (see below). I only changed the IP address for the MQTT server (my NAS) and the port (1883)

#include <AZ3166WiFi.h>
#include "MQTTClient.h"
#include "MQTTNetwork.h"
#include "Telemetry.h"

int status = WL_IDLE_STATUS;
int arrivedcount = 0;
bool hasWifi = false;

const char* mqttServer = "192.168.178.144";
int port = 1883;

void initWifi()
{
  Screen.print("IoT DevKit\r\n \r\nConnecting...\r\n");

  if (WiFi.begin() == WL_CONNECTED)
  {
    IPAddress ip = WiFi.localIP();
    IPAddress sn = WiFi.subnetMask();
    IPAddress gw = WiFi.gatewayIP();
    Screen.print(1, ip.get_address());
    Screen.print(2, sn.get_address());
    Screen.print(3, gw.get_address());
    hasWifi = true;
    Screen.print(4, "Running... \r\n");
  }
  else
  {
    Screen.print(1, "No Wi-Fi\r\n ");
  }
}

void messageArrived(MQTT::MessageData& md)
{
    MQTT::Message &message = md.message;

    char msgInfo[60];
    sprintf(msgInfo, "Message arrived: qos %d, retained %d, dup %d, packetid %d", message.qos, message.retained, message.dup, message.id);
    Serial.println(msgInfo);

    sprintf(msgInfo, "Payload: %s", (char*)message.payload);
    Serial.println(msgInfo);
    ++arrivedcount;
}

int runMqttExample() {
  char* topic = "mqtt-sample";
  MQTTNetwork mqttNetwork;
  MQTT::Client<MQTTNetwork, Countdown> client = MQTT::Client<MQTTNetwork, Countdown>(mqttNetwork);
  arrivedcount = 0;
  
  char msgBuf[100];
  sprintf(msgBuf, "Connecting to MQTT server %s:%d", mqttServer, port);
  Serial.println(msgBuf);

  int rc = mqttNetwork.connect(mqttServer, port);
  if (rc != 0) {
    Serial.println("Connected to MQTT server failed");
  } else {
    Serial.println("Connected to MQTT server successfully");
  }

  MQTTPacket_connectData data = MQTTPacket_connectData_initializer;
  data.MQTTVersion = 3;
  data.clientID.cstring = "mqtt-sample";
  data.username.cstring = "testuser";
  data.password.cstring = "testpassword";
  
  if ((rc = client.connect(data)) != 0) {
      Serial.println("MQTT client connect to server failed");
  }
  
  if ((rc = client.subscribe(topic, MQTT::QOS2, messageArrived)) != 0) {
      Serial.println("MQTT client subscribe from server failed");
  }
  
  MQTT::Message message;

  // QoS 0
  char buf[100];
  sprintf(buf, "QoS 0 message from AZ3166!");
  message.qos = MQTT::QOS0;
  message.retained = false;
  message.dup = false;
  message.payload = (void*)buf;
  message.payloadlen = strlen(buf)+1;
  rc = client.publish(topic, message);
  while (arrivedcount < 1) {
      client.yield(100);
  }
 
  // QoS 1
  sprintf(buf, "QoS 1 message from AZ3166!");
  message.qos = MQTT::QOS1;
  message.payloadlen = strlen(buf)+1;
  rc = client.publish(topic, message);
  
  while (arrivedcount < 2) {
      client.yield(100);
  }
  
  if ((rc = client.unsubscribe(topic)) != 0) {
      Serial.println("MQTT client unsubscribe from server failed");
  }
  
  if ((rc = client.disconnect()) != 0) {
      Serial.println("MQTT client disconnect from server failed");
  }
  
  mqttNetwork.disconnect();
  Serial.print("Finish message count: ");
  Serial.println(arrivedcount);
      
  return 0;
}

void setup() {
  //Initialize serial and Wi-Fi:
  Serial.begin(115200);
  initWifi();
  if(hasWifi)
  {
    // Microsoft collects data to operate effectively and provide you the best experiences with our products. 
    // We collect data about the features you use, how often you use them, and how you use them.
    send_telemetry_data_async("", "MQTTClientSetup", "");
  }
}

void loop() {
  Serial.println("\r\n>>>Enter Loop");

  if (hasWifi) {
    runMqttExample();
  }

  delay(5000);
}

b) What's running on the Synology (beside DSM) what should be reached by the Arduino?
I am running the Mosquitto server for Synology. See screenshot for version info. It is running on port 1883 and I can reach it using an MQTT client on my desktop.

**c) Please show us a snapshot of your browser accessing this resource on the Synology including the addressbar of your browser (the same you tried with a) ) **
See screenshot of MQTT client connection details as well as the broker messages while connected.
I also made a similar setup by installing Mosquitto on my desktop (in a docker container) running as a server. Then, running the MQTT client on my Arduino works fine and I can connect to the Mosquitto server on my desktop. However, the same setup does not seem to work with my NAS :frowning:

ok, at least we now know you are talking about MQTT communication.

Is there any PC client available to send mqtt messages? Can you try to send to the mqtt messages from your PC to your NAS? (this I wanted to see) - or any other working device beside the Arduino?

And vice versa: is there any MQTT Broker available beside your NAS? can you try to reach another service/server with the Arduino?

it's just to get a clear understanding, if the problem happens on the NAS side or Arduino.

Hi Noiasca,

Yes, my goal is to send temperature data from Arduino to my Domoticz (also on NAS) using the MQTT broker in between. I was able to run a similar setup: Arduino --> MQTT broker on my desktop, that worked fine.
The problem with my NAS is with networking however: I am not able to ping the Arduino and the Arduino is not able to reach my NAS: not via the MQTT client, but also not via a WebClient: I tried the sample webclient form my board and tried to reach the NAS on port 5000 (DSM) for example, just to check HTTP access. This is not working, so I concluded there must be a problem on (lower) networking level.

To answer your questions:
Is there any PC client available to send mqtt messages? Can you try to send to the mqtt messages from your PC to your NAS? (this I wanted to see) - or any other working device beside the Arduino?
Yes, I attached a (MQTT client) screenshot of the broker in my previous post - I can easily post a new message to the broker running on my NAS, even from my iphone! This works instantly when I enter the connection details: IP and port and that's it.

And vice versa: is there any MQTT Broker available beside your NAS? can you try to reach another service/server with the Arduino?
Yes, I setup the same Mosquitto broker on my desktop (in a Docker container), the I connected the Arduino (same sketch used as in my previous post) and this worked directly! So again: I am not worried about the Arduino nor the Arduino sketch - these have proven to work successfully. However, the problem seems to be on the networking side of my NAS (below the MQTT broker communication).

Do you have any idea how I could further test this/what to do? Thanks again!

Hello all,

I am one step closer to my goal: the problem is definately with routing in my network. On the NAS side, I added a static route rule:

root[member=41469]NAS[/member]:/etc# ip route show table static-table
192.168.178.200 via 192.168.178.1 dev eth0

In this way, my NAS can finally ping and traceroute to the Arduino. I can even do a curl from the NAS to the Arduino when I run a webserver on it. So: the route form NAS to Arduino seems OK now.
HOWEVER: I still cannot connect the Arduino to any web service running on my NAS (like MQTT or simply make a Http call to the NAS). This traffic form Arduino to NAS simply does not seem to reach the NAS somehow.
One would think that there may be a firewall or something in between my NAS and Arduino blocking the traffic. However, that makes no sense to me, since the Arduino is able to connect to my desktop (e.g. when running the same MQTT broker on my desktop, like I do on my NAS). This means the Arduino "traffic" is able to travel trhough my router (without firewall) to my desktop. So, why can't it then not do the same to my NAS? Is this something on the inbound NAS side or actually a problem with the Arduino? Then again: why is the Arduino able to ocnnect to other servers (not the NAS)? I am now even more confused... :slightly_frowning_face:
Is there anybody out there with some good network knowledge/skills who has any clue?

Get a copy of wireshark on your PC and examine the packets on the wire.

I suspect that your NAS is the problem child and it’s rejecting your Arduino packets.

palinkie:

    IPAddress ip = WiFi.localIP();

IPAddress sn = WiFi.subnetMask();
    IPAddress gw = WiFi.gatewayIP();
    Screen.print(1, ip.get_address());
    Screen.print(2, sn.get_address());
    Screen.print(3, gw.get_address());

What values do you get there?

Hi Erik,

These are the values that the Arduino gets:
LocalIP = 192.168.178.200 (I assigned this fixed in my router)
Subnet mask = 255.255.255.0
GatewayIP =192.168.178.1 (my router and default gateway)

What I don’t understand: for local (Arduin) to local (NAS) connection in the same subnet (192.168.178.x) the gateway should not be necessary. However, I can only get traffic from NAS to Arduino by making a static route on my NAS to set the default gateway to 192.168.178.1, specifically for the Arduino (192.168.178.200). That still does not make it possible for Arduino to reach NAS though. But at least it’s a start.... I was hoping to do something similar at Arduino site (static route to NAS), but that seems difficult (I don’t know how to set default gateway on the MXchip-based Arduino that uses slightly different libraries than the other Arduino boards? And my router is also hopeless (ziggo box without much routing options).
Do you have any clue?

@wildbill: I did a tcpdump on my NAS while the Arduino tried to reach it. However, I am only able to see a couple of ARP calls from the Arduino. NAS replies and after this, it is silent.
When I do the same thing using Wireshark on my desktop and set up a client on the Arduino to reach a web server on my desktop,I don’t see such ARP calls, but rather application requests and acknowledgements, like it should. No problem. So NAS is indeed spoiling the party, but I don’t have a clue how?

palinkie:
LocalIP = 192.168.178.200 (I assigned this fixed in my router)
Subnet mask = 255.255.255.0
GatewayIP =192.168.178.1 (my router and default gateway)

Nothing wrong with that. Another question: do you have DHCP active?

Yes, the DHCP is running on my router (ziggo box; the default gateway). See screenshot of the settings, which are in line with what the Arduino responds with when connecting to the WiFi network...

Hm, I see that the addresses you are using (192.168.178.200 and 192.168.178.1) also are in the DHCP range... Could it be that there is a conflict of addresses, because DHCP gave one of those numbers to some other device in your LAN?

My router is on 192.168.178.1 and that address is not within the dhcp range (10-255)
I also tried to assign 192.168.178.9 to the Arduino, but that did not make any difference
There are no IP conflicts / no other devices claiming the same IP or double Mac addresses etc
I think it is a matter of routing on NAS side, but I just don’t know what? :confused:

you can use Wireshark in promiscuous mode so you should be able to confirm that the Arduino is sending packets to the NAS but not getting a reply.