Go Down

Topic: Trouble with TextFinder and ESP8266 (Read 239 times) previous topic - next topic

dmacq

 

I'm way over my head here and don't see any easy way out - perhaps someone else can point me in the right direction.

Until recently, I've been using Textfinder to scape information off an Environment Canada weather api and it's worked perfectly for years. But recently, the api has demanded a fingerprint, and after much anguish, I managed to connect again using:

#include <WiFiClientSecure.h>
#include <ESP8266WebServer.h>

But my Textfinder routines no longer seem to read the data as it comes in. This one USED to work great for reading temperature and wind speed

 
---------------------------------------------

Serial.print   ("connecting to ");
  char* server = "weather.gc.ca";   // first part of the url
  Serial.println(server);
  if (!client.connect(server, httpsPort)) {
    Serial.println("connection failed");
    return;
  }

  if (client.verify(fingerprint, server)) {
    Serial.println("certificate matches");
  } else {
   // Serial.println("certificate doesn't match.. I don't know what this means ");
   // Serial.println("  ");
  }

  String url = "/wxlink/site_js/s0000458_e.js";   // the second part of the url
   // Serial.print("requesting URL: ");
   // Serial.println(url);

    client.print(String("GET ") + url + " HTTP/1.1\r\n" +
               "Host: " + server + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

  //Serial.println("Starting Connection to EnviroCanada");
 
if (client.connected()) {
     // String line = client.readStringUntil('\n');
       
    //  if (line == "\r") {
     
     
      if (finder.find("obTemperature") )  // this  has to be very exact. 
    {
      int temperature = finder.getValue();
     // Serial.print("Temperature is ");  //
    //  Serial.println(temperature);
 
    }
    else
      Serial.print("Could not find Temperature field");

      if (finder.find("obWindSpeed") )
    {
      WindSpeed = finder.getValue();
   //   Serial.print("WindSpeed is ");  //
    //  Serial.println( WindSpeed );
    }
    else
      Serial.print("Could not find WindSpeed field");
   
  }
   
  client.stop();  //Disconnect for next reload
  client.flush();//Serial.println(line);  // prints the whole page
  Serial.println( " ");   
  Serial.print(" Temperature is ");  //
  Serial.println(Temperature);
  Serial.print(" WindSpeed is ");  //
  Serial.println( WindSpeed );
  Serial.println( " "); 
  Serial.println("Ending  Connection to EnviroCanada");
  Serial.println( " ");

--------------------------------------

But no longer works at all .
 I thought this would be a matter of changing

WiFiClient client;
TextFinder  finder(client);

To
WiFiClient httpsClient;
TextFinder  finder(httpsClient);

and change all the other client references to httpsClient...

But it makes no difference and makes me think I've badly misunderstood the way Textfinder and client  works

Anyone have any ideas?

Thanks David


DrAzzy

And where is this "TextFinder" that you are using? Link to where you got the code from. Understanding this code will be key to solving your problem.

Also, have you tried printing out the string that you're searching, to make sure that it actually has the content you expect? That would be a good first step to understanding this; there may be some subtle difference in the formatting or content of the string that's causing problems.

Also, you posted a question about your project in the section for questions about installation/troubleshooting for the IDE or the board itself, despite the fact that it says at the top of the section not to do this. I have asked the moderators to move you post to the appropriate section.
ATTinyCore for x4/x5/x61/x7/x8/x41/1634/828/x313 megaTinyCore for the megaavr ATtinies - Board Manager:
http://drazzy.com/package_drazzy.com_index.json
ATtiny breakouts, mosfets, awesome prototyping board in my store http://tindie.com/stores/DrAzzy

dmacq

 

I'm way over my head here and don't see any easy way out - perhaps someone else can point me in the right direction.

Until recently, I've been using Textfinder to scape information off an Environment Canada weather api and it's worked perfectly for years. But recently, the api has demanded a fingerprint, and after much anguish, I managed to connect again using:

#include <WiFiClientSecure.h>
#include <ESP8266WebServer.h>

But my Textfinder routines no longer seem to read the data as it comes in. This one USED to work great for reading temperature and wind speed

 
Code: [Select]
--------------------------------------------

Serial.print   ("connecting to ");
 char* server = "weather.gc.ca";   // first part of the url
 Serial.println(server);
 if (!client.connect(server, httpsPort)) {
   Serial.println("connection failed");
   return;
 }

 if (client.verify(fingerprint, server)) {
   Serial.println("certificate matches");
 } else {
  // Serial.println("certificate doesn't match.. I don't know what this means ");
  // Serial.println("  ");
 }

 String url = "/wxlink/site_js/s0000458_e.js";   // the second part of the url
  // Serial.print("requesting URL: ");
  // Serial.println(url);

   client.print(String("GET ") + url + " HTTP/1.1\r\n" +
              "Host: " + server + "\r\n" +
              "User-Agent: BuildFailureDetectorESP8266\r\n" +
              "Connection: close\r\n\r\n");

 //Serial.println("Starting Connection to EnviroCanada");

if (client.connected()) {
    // String line = client.readStringUntil('\n');
       
   //  if (line == "\r") {
     
     
     if (finder.find("obTemperature") )  // this  has to be very exact.  
   {
     int temperature = finder.getValue();
    // Serial.print("Temperature is ");  //
   //  Serial.println(temperature);

   }
   else
     Serial.print("Could not find Temperature field");

     if (finder.find("obWindSpeed") )
   {
     WindSpeed = finder.getValue();
  //   Serial.print("WindSpeed is ");  //
   //  Serial.println( WindSpeed );
   }
   else
     Serial.print("Could not find WindSpeed field");
   
 }
 
 client.stop();  //Disconnect for next reload
 client.flush();//Serial.println(line);  // prints the whole page
 Serial.println( " ");  
 Serial.print(" Temperature is ");  //
 Serial.println(Temperature);
 Serial.print(" WindSpeed is ");  //
 Serial.println( WindSpeed );
 Serial.println( " ");  
 Serial.println("Ending  Connection to EnviroCanada");
 Serial.println( " ");

--------------------------------------


But no longer works at all .
I thought this would be a matter of changing

WiFiClient client;
TextFinder  finder(client);

To
WiFiClient httpsClient;
TextFinder  finder(httpsClient);[/code]

and change all the other client references to httpsClient...

But it makes no difference and makes me think I've badly misunderstood the way Textfinder and client  works

Anyone have any ideas?

Thanks David


UKHeliBob

Please do not send me PMs asking for help.  Post in the forum then everyone will benefit from seeing the questions and answers.

zoomkat

If you can still get the data, then you might just to look at that and see if there are changes that cause the textfinder application to fail.
Google forum search: Use Google Search box in upper right side of this page.
Why I like my 2005 Rio Yellow Honda S2000  https://www.youtube.com/watch?v=pWjMvrkUqX0

martinius96

Many times i am using only substring function :-)
And it is working too.
Arduino and website programmer

dmacq

Thanks for everyone's quick responses. TextFinder is a library for the Arduino I found at Github through  this link
https://playground.arduino.cc/Code/TextFinder/

I thought it was more widely used than it apparently is.

I am able to connect with the data from the websites, and read it into a string, but TextFinder just doesn't seem to be getting it or reading it any more.

Is there another library people use to read and scrape strings?  TextFinder was particularly useful since it could find a number value like temperature, following the word "Temperature".

I guess I could write something like that myself, but wow that would be a lot of work.

Sorry about sending this to the wrong forum, I haven't posted in a while.

Cheers

DmacQ

Go Up