HTTP 1.1 with YQL stopped working, error 400

Other Hardware Home Automation
1

I'm using an Arduino UNO R3 with a WiFly shield to connect to Yahoo Weather using a YQL request in an HTTP string. This is it:

client.println(F("GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%3D55903793%20and%20u%3D%27c%27&format=xml&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys HTTP/1.1"));

It worked as a charm and everything was AOK.
Then I left the project disconnected and in its box for 2 weeks as I had to check other projects.
Last week I connect it, wait for the data to appear and... nothing.
I debug and I find I'm receiving "HTTP 1.1 400 Invalid HTTP request."
I deleted the HTTP/1.1 part. Nothing. Different error, though. It froze.
Returned this part and URL encoded the whole string, resulting in this:

client.println(F("GET https%3A%2F%2Fquery.yahooapis.com%2Fv1%2Fpublic%2Fyql%3Fq%3Dselect+%2A+from+weather.forecast+where+woeid%3D55903793+and+u%3D%27c%27%26format%3Dxml%26env%3Dstore%3A%2F%2Fdatatables.org%2Falltableswithkeys HTTP/1.1"));

Same error 400.
Funny thing is, if I paste those requests (without the GET and HTTP/1.1 words, of course) in Chrome's address box, I get the results clean and clear, as the Arduino was receiving them before the failure.
Any ideas? I'm lost.

2

Any ideas? I'm lost.

You have connected to a server. The server name is NOT part of the GET request.

Most Arduinos can not handle https, so the http(s):// bit is not part of the GET request.

3

Well, well, well. I'll be damned! It worked! Thank you!
First, I removed the server name entirely, so the line changed to

GET /v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%3D55903793%20and%20u%3D%27c%27&format=xml&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys HTTP/1.1

But I got "HTTP/1.1 502 Cannot find server."

I just removed the https:// so the line changed to

GET query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%3D55903793%20and%20u%3D%27c%27&format=xml&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys HTTP/1.1

And there it is! The data at last!

What intrigues me the most is, why did it work for a year, then it stopped? In that year I had to change from

http://weather.yahooapis.com/forecastrss?w=55903793&u=c

to

http://xml.weather.yahoo.com/forecastrss?w=55903793&u=c

to the last one, and they all worked.

Anyway, thanks for your help!

4

A typical HTTP/1.1 request must minimally send two lines; the Host: line is always required with HTTP/1.1 for requests you're not sending through a proxy server. Example:

GET /standards/ HTTP/1.1
Host: www.w3.org

If you omit the Host: line you'll get a "400" error response.

With respect to why some of the other combinations are working (or were working), my best guess is that your WiFly shield is rewriting those requests or some device somewhere is being particularly lenient with your formatting. You can always prefix the URI with "http://www.hostname.com/" -- that's allowed -- but only with HTTP/1.0 are you allowed to omit the "Host:" header. And then again the exception to that rule is when sending a request through a proxy server, so it may be that your shield or something else is treating your requests like a proxy server would.

5

When I open the client, I do so with the host address. Also I have been working from the same place and the https:// string was working perfectly since july 16, according to my records. It suddenly started behaving like this. Not connecting and showing error 400 when connected.
What you comment leads me to suspect the turmoil in Yahoo! -because of those changes they are making- involve changes to the weather comms. A few months ago the service stopped altogether and after an uproar among developers they announced they were updating the servers and forgot to mention it.
Maybe it's time to change to OpenWeatherMap, but the need for keys and other details make me dubious about it.
Thanks for taking the time to comment! :slight_smile: