Arduino Enet get HTTP response

Hi,

I need to fetch an HTTP response. Not one char, but the entire thing.

So far I have this code, which works perfectly:

void httpRequest(byte *server, char *url)
{
  Client client(server, 80);
  
  if (client.connect())
  {
   // Serial.print("Sending req...");
    client.print("GET /");
    client.print(url);
    client.println(" HTTP/1.0");
    client.println();
    client.flush();
    client.stop();
  }
  else
  {
    analogWrite(9, 100);
    delay(5000);
    analogWrite(9, 0);
  }
}

Pin 9 is a buzzer, BTW.

So now I can send data to pages. But I also want the function to return what the page responded with. What is the best way to do this?

Thanks!

The example sketch for WebClient does that using this:

void loop()
{
  if (client.available()) {
    char c = client.read();
    [glow]// c has the data from the web server.[/glow]    Serial.print(c);
  }
  
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    for(;;)
      ;
  }
}

You probably will want to parse throught the headers, or look for "\r\n\r\n" if you want to skip the headers and get to the meaty HTML.

I would design the web page as simple as possible (have it just say 1 word on the page) and have the arduino look for that - if you are in control of the web server, that is...

I just did something like this for my weather monitoring system.

What I did was create a static global instance of Client, and a static global text buffer that's used for various interfaces.

/*
 * line_buff is used for both the serial and server connections to
 * accumulate characters into a complete line of text.  This
 * usually simplifies the code that processes the text.  To allow
 * overlap between data reading and other processing,  the buffer
 * is managed as a state machine.
 */
#define LINE_BUFF_SIZE 128
typedef enum {
      LINEBUFF_EMPTY,
      LINEBUFF_FILLING,
      LINEBUFF_READY,
      LINEBUFF_FULL
      } LINEBUFF_STATE;

byte linebuff_state = LINEBUFF_EMPTY;
char line_buff[LINE_BUFF_SIZE + 1];            // +1 for null terminator
byte line_cursor;

To manage the buffer, there's a routine that I call periodically while I'm doing other things. This gives me the option of "multitasking", although I eventually decided not to use it in this particular application: once the code starts the transaction with the server, it sticks with that until it either succeeds or fails.

/*
 * Read available data (if any) into the buffer.  If an event (like 
 * receiving a newline or overflowing the buffer) occurs,  change
 * linebuff_state accordingly.
 */
void buffer_server_response_bytes(void)
{
      byte c;

      if ((linebuff_state == LINEBUFF_READY)
                  || (linebuff_state ==  LINEBUFF_FULL))
            return;            // Already loaded.  Don't add more data.

      while (post_client.available())
            {
            c = post_client.read();
            content_bytes_received++;
            if (c == '\n')
                  {
                  linebuff_state = LINEBUFF_READY;
                  line_buff[line_cursor] = '\0';
                  break;
                  }
                  // Ignore CRs
            if (c != '\r')
                  {
                  line_buff[line_cursor++] = c;
                  linebuff_state = LINEBUFF_FILLING;
                  if (line_cursor == LINE_BUFF_SIZE)
                        {
                        linebuff_state = LINEBUFF_FULL;
                        line_buff[line_cursor] = '\0';
                        break;
                        }
                  }
            }
}

I have a routine similar to your code that sends the HTTP request

/*
 * Send a GET query to the server.
 * To reduce memory waste,  this routine allows the caller to split the
 * URL into a "base" (usually fixed) and a "parameters" (often variable)
 * part.  If either is not needed,  just pass NULL.
 * It also includes the constant parts of the GET command into the code.
 *
 * Returns 0 for "success",  non-zero on error
 */
int send_get(const char *url_base, const char *params)
{
      if (post_client.connect())
            {
            post_client.print("GET ");
            if (url_base)
                  post_client.print(url_base);
            if (params)
                  post_client.print(params);
            post_client.println(" HTTP/1.1");
#ifdef GODADDY_HOSTED
          // Virtual hosting requires the "Hostname: " header
            post_client.print("Host: ");
            strcpy_P(line_buff, server_name);
            post_client.println(line_buff);
#endif
            post_client.println("Accept: text/html");
                  // Send an empty line to end the transaction
            post_client.println();
            client_state = CLIENT_STATE_WAIT_RESPONSE;
            return 0;
            }
        else
            return 1;
}

The code that's conditionally compiled is worth noting in its own right: if you're going to deal with a site outside your own LAN, you may be required to go with HTTP 1.1, and add the "Host:" line to your request, so that servers hosting multiple sites on a single IP address (like godaddy, in my case) can direct it.

The mainline code then uses calls like this to get the response data as complete lines, instead of individual characters.

            case CLIENT_STATE_GETTING_RC:
                        if (post_client.connected())
                              {
                              buffer_server_response_bytes();
                              if ((linebuff_state == LINEBUFF_READY)
                                          || (linebuff_state ==  LINEBUFF_FULL))
                                    {      // We have a response line.  Get the result code.
                                    if (find_http_response_code() == 0)
                                          {      // Got response code.  Choose our next action.
                                          process_http_response_code();
                                          }
                                      else
                                          { // Oops: missing/garbled "HTTP/v.v" header.

The server interface code is also done as a state machine: every pass through loop(), it tries to gather up a complete line from the server, and advances the state as it finds the pieces of the response it's looking for.

In practice, I'm sure it's actually getting a complete line on every pass, but the design allows the application to interleave server interaction with other processing if your application demands it.

Ran