How to read specific HTML element in ESP8266 Client?

Projects Networking, Protocols, and Devices
1

I'm doing a bit of exercise, trying to get data/values from a website instead of using API or REST API. Turns out I get what I wanted but, the HTML response from the website is too long.

So I'm trying to get James Webb's Space Telescope (JWST) distance data from a website that I found. I used Chrome's developer tool to inspect the HTML element that I wanted and found it in the source file. Then I get to work on my ESP8266 to request the HTML page from the website.

image
image788×336 137 KB

I was able to get the main HTML page. I pasted it on a text editor and found the element's ID. Then I edited the code to display the element only.

tempsnip
tempsnip1131×371 37.7 KB

But it's very slow because prior to the element, it's reading a very long script line. This particular line is 60,000 characters long. Sometimes it resets the ESP8266 while reading it.

image
image1362×455 40.7 KB

So how do I exclude it, or rather exclude everything but the element that I want?

<span id="dissun">148881230</span>

Or is there any way that I could get the element by its ID like javascript?

The ESP8266 program (core v2.7.1):

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
const char *ssid = "my wifi ssid";
const char *password = "my wifi pass";

const char fingerprint[] PROGMEM = "77 9E 98 A7 38 60 72 0E D7 04 D5 24 0A 44 CD 03 EE 3F 96 CB";
WiFiClientSecure client;
const char* host = "theskylive.com";

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid,password);
  while(WiFi.status() != WL_CONNECTED){
    Serial.print(".");
    delay(300);
  }
  Serial.println("\nConnected: "+WiFi.localIP().toString());
  client.setTimeout(250);
  client.setFingerprint(fingerprint);
}

void loop(){
  
  if (client.connect(host,443)){
    Serial.print("[Connected]");
    
    client.println(request());
    
    while(client.connected()){
      if(!client.available())
      continue;
      
      String line = client.readStringUntil('\r');
      
      if(line.indexOf("dissun")<-1)
      continue;
      
      Serial.print("\nJWST from Sun: ");
      Serial.println(line.substring(line.indexOf("dissun")+8,line.indexOf("</span>",line.indexOf("</span>")+1)));
      break;
    }
    client.stop();
    Serial.println("[Disconnected]");
  }
  else{
    Serial.println("[Not connected]");
    client.stop();
  }
  delay(2000);
}

String request(){
  String l;
  l+="GET /jwst-tracker HTTP/1.1\r\n";
  l+="Host: " + String(host) + "\r\n";
  l+="Connection: close\r\n\r\n";
  return l;
}
2

try client.find or findUntil
https://www.arduino.cc/reference/en/language/functions/communication/stream/streamfind/

3

I'm not sure if this is the correct way but I got the results. Its still slow, I guess It still reads the whole html before the "dissun" element comes.

if(client.find("dissun")){
  line = client.readStringUntil('\r');
}
4

not the whole, only until it finds it. And it doesn't copy the characters into a buffer.

if you know that the searched string is not in first n characters, you can just read them without storing

5

Ok thanks, I guess thats the best I could go with, at least it stops the irregular resets.