Arduino Mega/ESP8266/API ThingSpeak

I am trying to get the weather using api.thingspeak.com.
I have signed up and can successfully go to the specified site and get the results I expect, using a browser. However, my program does not get the data I expect and I have been trying may diff approaches but none have been successful.

Using a browser - https://api.thingspeak.com/apps/thinghttp/send_request?api_key=My_API_Key
results - 49° (current temp in my city)

I never receive this info using Arduino Mega & ESP8266

I can get the time from google by reading the gmt time returned in the header.
Does anyone have any suggestions as I feel I have tried everything.

Here is a simplified program I am using for troubleshooting:

String inputString = "";
inputString.reserve(300);

void getW() {
  cmd = "AT+CIPSTART=\"TCP\",\"";
  cmd += "api.thingspeak.com"
  cmd += "\",80";
  
  DEBUG.println("from 111==========================");
  ESP8266.println(cmd);  //send command to device
 while (Serial1.available()) {
    // get the new byte:
    char inChar = (char)Serial1.read();
    // add it to the inputString:
    inputString += inChar;}
  DEBUG.println(inputString);  
  inputString = "";
  DEBUG.println("done 111============================");
  delay(3000);
  
  cmd =  "GET https://api.thingspeak.com/apps/thinghttp/send_request?api_key=MyKey headers=false"; 
  //cmd += "Connection: close"; 

  DEBUG.println("from 222=================================");
  ESP8266.print("AT+CIPSEND=");
  ESP8266.println(cmd.length());
  while (Serial1.available()) {
    // get the new byte:
    char inChar = (char)Serial1.read();
    // add it to the inputString:
    inputString += inChar;}
  DEBUG.println(inputString);  
  inputString = "";
  DEBUG.println("done 222================================");
  delay(3000);

  DEBUG.println("from 333=======================");
  ESP8266.println(cmd);
     while (Serial1.available()) {
    // get the new byte:
    char inChar = (char)Serial1.read();
    // add it to the inputString:
    inputString += inChar;}
  DEBUG.println(inputString);  
  inputString = "";
  DEBUG.println("done 333=============================");
  delay(3000);
 
   DEBUG.println("from 444=================================");
   ESP8266.println("AT+CIPCLOSE"); // Close the TCP connection
   while (Serial1.available()) {
    // get the new byte:
    char inChar = (char)Serial1.read();
    // add it to the inputString:
    inputString += inChar;}
  DEBUG.println(inputString);  
  inputString = "";
  DEBUG.println("done 444===================================");
  delay(3000);
}

//=======================================================================
void loop() 
{
getW();
delay(10000);
}

And this is the results:

Waiting...-->AT+CIFSR

+CIFSR:STAIP,"192.168.1.15"
+CIFSR:STAMAC,"82:66:ff:ff:82:66"

OK

Waiting...-->AT+GMR

AT version:1.3.0.0(Jul 14 2016 18:54:01)
SDK version:2.0.0(656edbf)
compile time:Jul 19 2016 18:44:44
OK

============================
from 111==========================

done 111============================
from 222=================================
AT+CIPSTART="TCP","api.thingspeak.com",80

CONNECT

OK

done 222================================
from 333=======================
AT+CIPSEND=97


OK
> 
done 333=============================
from 444=================================


busy s...

Recv 97 bytes

SEND OK

done 444===================================

Do you have a USB to TTL adapter so you can hook up your PC direct to your ESP8266? Then you can try the AT commands interactively and see the response.

Some AT examples at at_example_0020000903 · espressif/ESP8266_AT Wiki · GitHub. Its not clear if you need to put > before your data or if thats a prompt from the ESP8266

Some discussion at http://bbs.espressif.com/viewtopic.php?t=116 from people with same issues.

I only tried interfacing an ESP via AT commands once (this was before ESPArduino). It was terrible. Some part was due to bad connections, some part due to quickly changing firmware at that point, so it might be better now, but I still wouldn't do it again. Isn't there a lib for that, nowadays?

I would consider programming the ESP itself. It is faster and more powerful than a Mega anyway. Then either connect your Mega via I2C or cut it completely and just use an I2C port extender if you need all the ports.

I did finally figure out what was gong on.
When using the site hurl.it to test get commands, I found that I was missing part of the cmd line. The new line is as follows:

cmd =  "GET https://api.thingspeak.com/apps/thinghttp/send_request?api_key=MY_KEY\r\nQUERYSTRING api_key: MY_KEY";

Then through pure desperation I tried sending a blank line after sending the get command. Once I sent this, I could get the response I needed.
here is what I added between parts 3 & 4 After the data is received, the connection is closed and there is no reason to try and close it, it just fails so that part can be omitted

   DEBUG.println("from 3.5 =================================");
     ESP8266.println(""); // 
   delay(1000);
    if (Serial1.find(":"))
    {while (Serial1.available()) {
       char inChar = (char)Serial1.read();
      inputString += inChar;}
  String needle = "CL";
  int foundpos = -1;
  for (int i = 0; i <= inputString.length() - needle.length(); i++) {
    if (inputString.substring(i,needle.length()+i) == needle) 
      {foundpos = i; }}
  DEBUG.println("foundpos = " + String(foundpos));
  DEBUG.println(inputString.substring(0, foundpos)); }  
  inputString = "";
  DEBUG.println("done 3.5===================================");
  delay(1000);}

finally I get

from 111==========================

done 111============================
from 222=================================
AT+CIPSTART="TCP","api.thingspeak.com",80

CONNECT

OK

done 222================================
from 333=======================
GET https://api.thingspeak.com/apps/thinghttp/send_request?api_key=xxxxxxxxxxxxxx
QUERYSTRING api_key: xxxxxxxxxxxxxxxxxx
AT+CIPSEND=122


OK
> 
done 333=============================
from 3.5 =================================
foundpos = 8
Overcast
done 3.5 ===================================