[SOLVED] ESP-01 (WiFiEsp) Unable to get request response from some sites

Using Arduino Networking, Protocols, and Devices
#1

Hi.
I'm developing a monitoring system which will collect some information and send it to my server via Wi-Fi.

I'll use arduino mega, ESP-01 and WiFiEsp library.
I've no problem with wiring and code, I'm able to send all required information to the server (it receives all data) but there is a trouble with the server response.
Currently I'm using node js + express as a dev server and it returns simple string to each request but using WiFiEsp I'm unable to get this response.

Then I've tried to send GET requests to different sites and find out that I could get request only from a small part of these servers.

#include "WiFiEsp.h"
#define HAVE_HWSERIAL1
// Emulate Serial1 on pins 6/7 if not present
#ifndef HAVE_HWSERIAL1
#include "SoftwareSerial.h"
SoftwareSerial Serial1(6, 7); // RX, TX
#endif

char ssid[] = "xxxx";            // your network SSID (name)
char pass[] = "xxxx";        // your network password
int status = WL_IDLE_STATUS;     // the Wifi radio's status

char server[] = "arduino.ua";

// Initialize the Ethernet client object
WiFiEspClient client;

void setup()
{
  // initialize serial for debugging
  Serial.begin(115200);
  // initialize serial for ESP module
  Serial1.begin(115200);
  // initialize ESP module
  WiFi.init(&Serial1);

  // check for the presence of the shield
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
    // don't continue
    while (true);
  }

  // attempt to connect to WiFi network
  while ( status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network
    status = WiFi.begin(ssid, pass);
  }

  // you're connected now, so print out the data
  Serial.println("You're connected to the network");
  
  printWifiStatus();

  Serial.println();
  Serial.println("Starting connection to server...");
  // if you get a connection, report back via serial
  if (client.connect(server, 80)) {
    Serial.println("Connected to server");
    // Make a HTTP request
    client.println("GET / HTTP/1.1");
    client.println("Host: arduino.ua");
    client.println("Connection: close");
    client.println();
  }
}

void loop()
{
  delay(500);
  // if there are incoming bytes available
  // from the server, read them and print them
  while (client.available()) {
    char c = client.read();
    Serial.write(c);
  }

  // if the server's disconnected, stop the client
  if (!client.connected()) {
    Serial.println();
    Serial.println("Disconnecting from server...");
    client.stop();

    // do nothing forevermore
    while (true);
  }
}


void printWifiStatus()
{
  // print the SSID of the network you're attached to
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print your WiFi shield's IP address
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // print the received signal strength
  long rssi = WiFi.RSSI();
  Serial.print("Signal strength (RSSI):");
  Serial.print(rssi);
  Serial.println(" dBm");
}

Response

[WiFiEsp] Initializing ESP module
[WiFiEsp] Initilization successful - 1.5.4
Attempting to connect to WPA SSID: XXXX
[WiFiEsp] Connected to XXXX
You're connected to the network
SSID: XXXX
IP Address: 192.168.0.100
Signal strength (RSSI):-622 dBm

Starting connection to server...
[WiFiEsp] Connecting to arduino.ua
Connected to server
HTTP/1.1 200 OK
Date: Tue, 09[WiFiEsp] TIMEOUT: 1422

As you can see it is only a part of the server response..
In the same time I can get full response from servers: itc.ua, www.tecposter.com and probably some other.

Could you suggest some solution?

#2

What, exactly, is at / on arduino.au?

Look at the ethernet examples. When getting response data, you need to look at both connected() and available(). Looking at only one of them is not sufficient.

#3

PaulS, arduino.ua it is just a local website about arduino, I've used it to test GET requests.
As I said previously, I had used different websites for testing (GET request to root should return index page) and only part of that test succeeded.

I've just checked examples for Ethernet, it looks almost like my test code

Like in my example they use client.connected() to check whether the client is connected and stop it in other case.

Could you explain what do you mean "to look at both connected() and available()"?
Thank you.

#4

Could you explain what do you mean "to look at both connected() and available()"?

You may have client.connected() return true, while client.available() returns 0, because the server has not finished sending data, but the client has read everything that has arrived.

   while(client.connected())
   {
      // There is a client connected. Is there anything to read?
      while(client.available() > 0)
      {
         char c = client.read();
         Serial.print(c);
      }
      Serial.println();
      Serial.println("Waiting for more data...");
      delay(100); // Have a bit of patience...
   }
#5

There were another issue.
Connection become closed immediately after request (due to client.println("Connection: close"); ) so after checking if client.connected() code fall into infinity while.

Main idea: we should check for available data always, does not matter if connection closed or not.

PaulS, thank you for your help.

#6