WiFiClient.connect - intermittent failures

Hi all,

I am using the MKR1000 to build a sensor that regularly (every 10 secs) sends its measurements to an HTTP web service.

This seems to work ok for a while, but after a few successful sends (data is also correctly received), client.connect returns false (typically several times in a row) then sometimes it works again etc.

In average only 50% (!!!) of the send attempts do work properly and there does not seem a pattern.

The code is the same as in the samples, i.e.
...
if (client.connect(server, 80)) {
client.println("PUT /api/readings HTTP/1.1");
...

If client.connect() fails, WiFi.status() still equals WL_CONNECTED.

I also tried to disable all serial debug output but no avail.

Are there any other options to debug client.connect issues?
Others have reported the same or similar issues. Is this a known problem with the WiFi101 library?

Thanks for any advice!
If I dont get this to work more reliably, the project will fail :frowning:

You didn't say what web service you are connecting to but some have a minimum post time.

EG don't post any less than every 15 seconds (thingspeak)

Make sure you are not posting too frequently as many services will reject connections.

A little more detail would be great too eg. the code you are using (minus passwords/keys etc)

Thanks for your reply, Ballscrewbob!
I am currently reducing my sketch to the max (i.e. https://www.arduino.cc/en/Tutorial/Wifi101WiFiWebClientRepeating )

Looking good so far. Will keep you posted ...

OK. Found something: As long as this code in the main loop is not uncommented, everything works ok:

while (client.available()) {
char c = client.read();
}

However, If i comment this out (and do not read the resonse), the MKR1000 starts behaving strangely.

Looks like client.flush() is gonna be my new friend :wink:

Hi,
I am making a basic WiFi hello world program to let me know when MKR1000 is connected and disconnected from WiFi and when it is connected to client and when disconnected. but its not working. i am using WiFi101 library and modifying the basic AP example. i put serial.println at different steps to show me the statuses. can someone share a basic code to show me when MKR1000 (as AP) is connected via WiFi and when disconnected and also when a client is connected and disconnected.

Hi everyone,

I've been working on some changes to remove the need to call client.flush(), as well as some other improvements.

If you're interested in trying these changes before they are merged and released into a new version of the WiFi101 library please try out the following pull request:

https://github.com/arduino-libraries/WiFi101/pull/77

We'd appreciate any feedback and testing you could provide.

Thanks.

Going to try this lib with thingspeak as that is the one that always crashes with the MKR.

Thanks Sandeep

EDIT

Over 20 hours and counting sending data to thingspeak without a stall...Fingers crossed that it continues.

You are the man Sandeep.

Going to leave this running as 2 days was the max I ever got before but usually in the 2 hours to about 24 hours was average.

Hi @Ballscrewbob,

Any more news on your tests? Were you able to run your sketch for more than 2 days?

Thanks.

@ Sandeep

It still fails at random periods.
However it did seem a little more stable with almost 3 days being a record.

Mine was 24 hrs max normally but I can easily get 48 hrs or more now.

Still have your JSON listed in board manager and that's the only change otherwise so something in there made a slight difference for the better.

If you check my posts on the MKR topics you can see I have tested the living daylights out of mine with even simple sketches and all of them fail at some point even with serial disabled / not used.
As I have said in some of my other posts I can sometimes make it re-start by connecting to serial but that's just because it polls the USB.

There is no pattern at all to it as I tend to be quite methodical in how I approach things being a millwright as my primary trade.

Not getting as much Arduino time as I would like right now having moved to a new house that needs lots of small jobs doing.

Hi @Ballscrewbob,

You've been very busy on the forum! Thanks for helping out the community!

What would really help is if you select one or two of the simplest sketches that are failing for you, then I can try to reproduce the issues. We can then bubble them up to Github issues (Issues · arduino-libraries/WiFi101 · GitHub) where they have more visibility than the forums.

Hi Sandeep.

Included two that I know fail here under Windows 7 Pro X64 on an AMD quad core system.
There is nothing fancy about my hardware but if you need a hardware list let me know and I will include it for completeness.

This one fails but seems to run well for quite some time before it does (2-3 days with your JSON)
This is also a sketch that if I leave the CREATE serial window open actually causes a full blown BSOD when the MKR fails.

The next one is the one that gives me most problems with varied fail times.
Wanted to use this as a base for more advanced sensors but even with a simple DHT sensor it craps out.
I have another version of this with all serial removed that still fails.

  /*
  This sketch is a combination of ADAFruits DHT sketch, WiFi101 Webclient
  and The arduino example from ThingSpeak
  Modified by Stephen Borsay for the MKR1000, feel free to use
  added a brute force float string handler to address package size issue
   */
  
  #include <WiFi101.h>
  #include "DHT.h"
  
  #define DHTPIN 3    // what pin we're connected to, pin1 is 5th pin from end
  
  // Uncomment whatever DHT sensor type you're using!
  //#define DHTTYPE DHT11  // DHT 11
  #define DHTTYPE DHT21  // DHT 21
  //#define DHTTYPE DHT22  // DHT 22
  
  DHT dht(DHTPIN,DHTTYPE);
  
  String apiKey ="REMOVED";  // api from ThingSpeak
  
    char ssid[] = "REMOVED";         // your network SSID (name)
    char pass[] = "REMOVED";    // your network password
  int keyIndex = 0;     // your network key Index number (needed only for WEP)
  
  
  int status = WL_IDLE_STATUS;
  // if you don't want to use DNS (and reduce your sketch size)
  // use the numeric IP instead of the name for the server:
  //IPAddress server(74,125,232,128);  // numeric IP for Google (no DNS)
  char server[] = "api.thingspeak.com";    // name address for Google (using DNS)
  
  // Initialize the Ethernet client library
  // with the IP address and port of the server
  // that you want to connect to (port 80 is default for HTTP):
  WiFiClient client;
  
  void setup() {
    //Initialize serial and wait for port to open:
    Serial.begin(9600);
    while (!Serial) 
    {
      ; // wait for serial port to connect. Needed for native USB port only
    }
  
    // check for the presence of the shield:
    if (WiFi.status() == WL_NO_SHIELD) {
      Serial.println("WiFi shield not present");
      // don't continue:
      while (true);
    }
  
    // attempt to connect to Wifi network:
    while (status != WL_CONNECTED) 
    {
      Serial.print("Attempting to connect to SSID: ");
      Serial.println(ssid);
      //Connect to WPA/WPA2 network.Change this line if using open/WEP network
      status = WiFi.begin(ssid, pass);
  
      // wait 10 seconds for connection:
      delay(10000);
    }
    
    Serial.println("Connected to wifi");
    printWifiStatus();
    
  }
  
  void loop() {
  
     // Wait a few seconds between measurements.
    delay(20000);
  
    //prefer to use float, but package size or float conversion isnt working
    //will revise in future with a string fuction or float conversion function
  
    float h = dht.readHumidity();
    // Read temperature as Celsius (the default)
    float t = dht.readTemperature();
    // Read temperature as Fahrenheit (isFahrenheit = true)
    float f = dht.readTemperature(true);
  
    // Check if any reads failed and exit early (to try again).
    if (isnan(h) || isnan(t) || isnan(f))
    {
      Serial.println("Failed to read from DHT sensor!");
      return;
    }
  
    char  buffer[27];
  
    // Compute heat index in Fahrenheit (the default)
    int hif = dht.computeHeatIndex(f, h);
    // Compute heat index in Celsius (isFahreheit = false)
    int hic = dht.computeHeatIndex(t, h, false);
  
    Serial.print("Humidity: ");
    Serial.print(h);
    Serial.print(" %\t");
    Serial.print("Temperature: ");
    Serial.print(t);
    Serial.print(" *C ");
    Serial.print(f);
    Serial.print(" *F\t");
    Serial.print("Heat index: ");
    Serial.print(hic);
    Serial.print(" *C ");
    Serial.print(hif);
    Serial.println(" *F\n");
  
      Serial.println("\nStarting connection to server..."); 
    // if you get a connection, report back via serial:
    if (client.connect(server, 80)) {
      Serial.println("connected to server");
  
  // brute force string handling converter for int to float
  //normal flote data types wont work, feel free to try though
  
   int h1 =  (h * 100);
   int t1 =  (t * 100);
   int f1 =  (f * 100);
  
  
            client.print(F("POST "));
            client.print("/update?key=apiKey&field1=" 
            +               (String)(h1/100) +'.' +(String)((h1%100)/10) +(String)(h1%10)
            +  "&field2=" + (String)(t1/100) +'.' +(String)((t1%100)/10) +(String)(t1%10)
            +  "&field3=" + (String)(f1/100) +'.' +(String)((f1%100)/10) +(String)(f1%10)
            +  "&field4=" + (String) hic
            +  "&field5=" + (String) hif
                                     );
                                        
            String tsData = "field1="   //need the length to give to ThingSpeak
            +              (String)(h1/100) +'.' +(String)((h1%100)/10) +(String)(h1%10)
            +  "&field2=" +(String)(t1/100) +'.' +(String)((t1%100)/10) +(String)(t1%10)
            +  "&field3=" +(String)(f1/100) +'.' +(String)((f1%100)/10) +(String)(f1%10)
            +  "&field4=" +(String) hic
            +  "&field5=" +(String) hif
          ; 
  
  
            client.print("POST /update HTTP/1.1\n");  
            client.print("Host: api.thingspeak.com\n");
            client.print("Connection: close\n");
            client.print("X-THINGSPEAKAPIKEY: " + apiKey + "\n");
            client.print("Content-Type: application/x-www-form-urlencoded\n");
            client.print("Content-Length: ");
            client.print(tsData.length());  //send out data string legth to ts
            client.print("\n\n");
            client.print(tsData);
            client.stop();
            delay(1000);
      } 
  
  }
  
  
  void printWifiStatus() {
    // print the SSID of the network you're attached to:
    Serial.print("SSID: ");
    Serial.println(WiFi.SSID());
  
    // print your WiFi shield's IP address:
    IPAddress ip = WiFi.localIP();
    Serial.print("IP Address: ");
    Serial.println(ip);
  
    // print the received signal strength:
    long rssi = WiFi.RSSI();
    Serial.print("signal strength (RSSI):");
    Serial.print(rssi);
    Serial.println(" dBm");
  }

Hi @Ballscrewbob,

Thanks for the info.

Could you please create a new issues on Github for them?

For the RTCZero, please report it here: GitHub - arduino-libraries/RTCZero: RTC Library for SAMD21 based boards.

The Thing Speak example should be reported here: GitHub - arduino-libraries/WiFi101: Wifi library for the Arduino WiFi 101 Shield

I'm asking you to report them, because you will get notified of any updates through Github instead of reposting on the forum.

Regarding the ThinkSpeak issue. Have you tried removing the DHT out of the equation by just sending constant values? As well checking the value of "WiFi.status()" in loop to see if it's still connected to the access point?

It would be nice to isolate if it's a WiFi101 or DHT issues. Also, could you please clarify what you mean by it "fails".

Hi again Sandeep.

Noticed you had updated the pull so have switched to your latest.

Did put a request in for the RTC but as it also involves "create" and that is still a closed/ private section I only put the request in on the RTC git.

Q. Should I still include the std MKR JSON or just use yours as right now I have both in the prefs. ?

And thank you for taking so much time over these issues.

Q. Should I still include the std MKR JSON or just use yours as right now I have both in the prefs. ?

Which JSON URL are you referring too?

Before opening an issue for the DHT sketch you provided. We need to isolate if it's a DHT issue or a WiFi101 issue. Could you please try my suggestions from my last post:

Regarding the ThinkSpeak issue. Have you tried removing the DHT out of the equation by just sending constant values? As well checking the value of "WiFi.status()" in loop to see if it's still connected to the access point?

This is will help us figure out to the right place to open the issue. Also, could you please expand to what you mean by "fails".

Thanks again for your help in resolving these issues.

Sandeep you gave me a JSON to try

"http://downloads.arduino.cc/PR/samd/package_samd-b84_index.json"

But when I was looking at some of your GIT comments I noticed you had updated to

"http://downloads.arduino.cc/PR/samd/package_samd-b86_index.json"

I added that so that right now I have the following.

"boardsmanager.additional.urls=http://downloads.arduino.cc/packages/package_mkr1000_index.json,http://arduino.esp8266.com/stable/package_esp8266com_index.json,http://downloads.arduino.cc/PR/samd/package_samd-b86_index.json"

And BTW this seems to be working quite well with the DHT sketch and has not failed for over 36 hours since I added your current JSON.

My concern is that I have two packages for the MKR and I was unsure if I should remove one ?

Am going to wait and see if this newer JSON fails before I change to a TMP based sensor to follow your other advice. I have no idea what you changed in "samd-86" but I have a sneaking suspicion that you may have nailed it.

EDIT / UPDATE

Now over 48 hours and zero issues. I have had the regular IDE serial console open all this time too so I am going to turn that off for another test run... Wish me luck...

OK new (repeat issue) ?

If I leave the serial monitor open it works superbly no problem for over 48 hours (going to class that as a WIN)

However if I disable ALL serial commands it fails quite quickly.

Currently running with serial enabled but no serial output enabled.

UPDATE to Sandeepmistry...

Thank you.

So far well over 48 hours and no failures.

Using the JSON listed above but I have to leave

Serial.begin(9600);

In or else the sketch will still fail randomly even if there are no other serial output statements in the sketch.
Its a tiny price to pay to have this thing actually working properly.

Currently working on bringing back up my workstation for my hobbies after moving house and will soon be able to deploy some better sensors thanks to you.

Hi @Ballscrewbob,

Excellent detective work!

Please go ahead and open an issue for the behaviour you see without the "Serial.begin(9600);" here: Issues · arduino/ArduinoCore-samd · GitHub. Can you reproduce the issue with a much simpler sketch like the Blink example?

"http://downloads.arduino.cc/PR/samd/package_samd-b86_index.json" maps to the following pull request: Add USB send timeout by sandeepmistry · Pull Request #152 · arduino/ArduinoCore-samd · GitHub which has since been replaced by Add USB send timeout (take 2) by sandeepmistry · Pull Request #154 · arduino/ArduinoCore-samd · GitHub

Please also include the pull request number along with the JSON, it's easier to correlate this way.

Pull request #154 and #152 add a send timeout for each USB CDC write (Serial on the MKR100 is SerialUSB) . Previously, it would wait for the PC side to acknowledge the data indefinitely.

Just curious were you opening and closing the Serial Monitor during your tests? Or was the PC going to sleep?

@ Sandeep

Negative on open and close serial monitor as I left it open the whole time when serial statements were enabled.

Left it off the whole time when serial statements were disabled. (apart from the serial begin command)

This computer is set to NOT allow any types of sleep or hibernation modes including individual power saves on USB or network etc. Only thing that runs is screen saver but that's set to also remain on full power.
Pretty much maximum power always as I want it available at any time and don't want any delays when I come to use it.

Will play later with simpler sketches and take it from there before I report to GIT.
Updated to b88 JSON from pull 154 in prefs too.

I could not have gotten this far without your help.