Go Down

Topic: Very new to Arduino - Project using ESP8266 to fetch weather data  (Read 1 time) previous topic - next topic

blnx94

Hi there,

First I just want to apologize as I am very new to Arduino coding so please bare with me..

I am working on a project that requires the ESP8266 to connect to a weather website (specifically to the link : http://weather.gc.ca/forecast/hourly/on-143_metric_e.html)
and extract the 24 hour forecast (i.e., the time and corresponding temperature).

From there it would be able to read and output the lowest temperature in that 24h forecast.

I have attached the code I am currently using which outputs into the serial monitor the entire webpage as a text.

However, I am confused on what to do next.

Any help would be greatly appreciated.

Thanks!




PaulS

Quote
However, I am confused on what to do next.
Code: [Select]
  while(client.available()){
    String line = client.readStringUntil('\r');
    Serial.print(line);
  }

Something comes back from the server and gets printed.

The next step is to determine is that line contains anything useful. If it does, extract it.
The art of getting good answers lies in asking good questions.

blnx94

Hi Paul,

I'm not too familiar with coding so not sure how I would be go about doing that, it's where I'm stuck in fact.
Would you be able to provide me with a resource where I can learn this?

I don't want to ask for a code to be written - would much rather do it myself but no idea where to start/what existing syntax to use.

Thank you for your response

PaulS

Quote
I don't want to ask for a code to be written - would much rather do it myself but no idea where to start/what existing syntax to use.
The search term is parsing. You need to see if the String contains (has an indexOf() that is not negative) something useful, like "temperature = 34;" or "chance of rain = Most definitely;".
If the name part is present, then use indexOf() again to find the start of the interesting part (after the =, for instance" and to find the end of the interesting part (the ; in my example), and create a substring of the part in between.

It would be easier to give advice about parsing if we had any idea that the data to be parsed looked like.
The art of getting good answers lies in asking good questions.

xl97

it would be much easier/faster if you could tap into some personal webserver and call/request a PHP script..

that does the scraping & parsing for you....

not really helpful if you want go down the ESP direct connect route though..


Anywho-

"<table class="table table-striped table-hover table-condensed wxo-media">"

is the beginning of the HTML table that holds the data you want to scrape & parse..


unless you want the headers for some reason..

It looks the data that has any meaning to you starts here:




Code: [Select]
<tr>
<td class="text-center" headers="header1">16:00</td>

<td class="text-center" headers="header2">3</td>

<td class="media" headers="header3">
<span class="pull-left">
<img class="media-object" width="35" height="35" alt="Sunny" src="/weathericons/small/00.png">
</span>
<div class="media-body">
<p>Sunny</p>
</div>
</td>

<td class="text-center" headers="header4">Nil</td>

<td class="text-center" headers="header5">
<abbr title="Southwest">SW</abbr>10<br>
</td>
</tr>



So:
TD[0]  is the time
TD[1]  is the temp
TD[2]  is the conditions
TD[3]  is the LOP?
TD[4]  is the wind

So you can ignore all table tags..
any colgroup stuff
any tr > th tags..

actually starting at tbody is where the real content starts...

so any tr >> td class =  has content you want..

no other TD's..







xl97

HTML DUMP of table only:

Code: [Select]

<table class="table table-striped table-hover table-condensed wxo-media">
    <colgroup>
      <col />
      <col />
      <col />
      <col />
      <col />
    </colgroup>
    <thead>
      <tr>
        <th id="header1" class="wxo-tb-head">Date/Time<br />(<abbr title="Eastern Daylight Time">EDT</abbr>)</th>
        <th id="header2" class="wxo-tb-head">
          <abbr title="Temperature">Temp.</abbr>
          <br />(<abbr title="degrees Celsius">&deg;C</abbr>)</th>
        <th id="header3" class="wxo-tb-head">Weather Conditions</th>
        <th id="header4" class="wxo-tb-head">
          <abbr title="Likelihood of precipitation">LOP<sup id="fn1-rf">
              <a class="fn-lnk" href="#fn1"><span class="wb-inv">Footnote </span>†</a>
            </sup>
          </abbr>
        </th>
        <th id="header5" class="wxo-tb-head">Wind<br />(<abbr title="kilometres per hour">km/h</abbr>)</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <th class="wxo-th-bkg" colspan="5">23 March 2017</th>
      </tr>
      <tr>
        <td headers="header1" class="text-center">16:00</td>
        <td headers="header2" class="text-center">3</td>
        <td headers="header3" class="media"><span class="pull-left"><img class="media-object" height="35" width="35" src="/weathericons/small/00.png" alt="Sunny" /></span><div class="media-body">
            <p>Sunny</p>
          </div>
        </td>
        <td headers="header4" class="text-center">Nil</td>
        <td headers="header5" class="text-center">
          <abbr title="Southwest">SW</abbr> 10<br />
        </td>
      </tr>
      <tr>
        <td headers="header1" class="text-center">17:00</td>
        <td headers="header2" class="text-center">3</td>
        <td headers="header3" class="media"><span class="pull-left"><img class="media-object" height="35" width="35" src="/weathericons/small/00.png" alt="Sunny" /></span><div class="media-body">
            <p>Sunny</p>
          </div>
        </td>
        <td headers="header4" class="text-center">Nil</td>
        <td headers="header5" class="text-center">
          <abbr title="East">E</abbr> 10<br />
        </td>
      </tr>
      <tr>
        <td headers="header1" class="text-center">18:00</td>
        <td headers="header2" class="text-center">2</td>
        <td headers="header3" class="media"><span class="pull-left"><img class="media-object" height="35" width="35" src="/weathericons/small/01.png" alt="Mainly sunny" /></span><div class="media-body">
            <p>Mainly sunny</p>
          </div>
        </td>
        <td headers="header4" class="text-center">Nil</td>
        <td headers="header5" class="text-center">
          <abbr title="East">E</abbr> 10<br />
        </td>
      </tr>
   
    </tbody>
  </table>



too long so had to be truncated.....

Go Up