Ethercard lib. and webclient example with big sites

Projects Programming
1

Hi all,

I've the ENC28J60 board up and running and I'm planning to use the webclient example to retrieve the weather forecast on regular basis using the following URL: http://gps.buienradar.nl/getrr.php?lat=52&lon=4

// Demo using DHCP and DNS to perform a web client request.
// 2011-06-08 <jc@wippler.nl> http://opensource.org/licenses/mit-license.php

#include <EtherCard.h>

// ethernet interface mac address, must be unique on the LAN
static byte mymac[] = { 0x74,0x69,0x69,0x2D,0x30,0x31 };

byte Ethernet::buffer[700];
static uint32_t timer;

char website[] PROGMEM = "gps.buienradar.nl";

// called when the client request is complete
static void my_callback (byte status, word off, word len) {
  Serial.println(">>>");
  Ethernet::buffer[off+300] = 0;
  Serial.print((const char*) Ethernet::buffer + off);
  Serial.println("...");
}

void setup () {
  Serial.begin(57600);
  Serial.println("\n[webClient]");

  if (ether.begin(sizeof Ethernet::buffer, mymac) == 0) 
    Serial.println( "Failed to access Ethernet controller");
  if (!ether.dhcpSetup())
    Serial.println("DHCP failed");

  ether.printIp("IP:  ", ether.myip);
  ether.printIp("GW:  ", ether.gwip);  
  ether.printIp("DNS: ", ether.dnsip);  

  if (!ether.dnsLookup(website))
    Serial.println("DNS failed");
    
  ether.printIp("SRV: ", ether.hisip);
}

void loop () {
  ether.packetLoop(ether.packetReceive());
  
  if (millis() > timer) {
    timer = millis() + 5000;
    Serial.println();
    Serial.print("<<< REQ ");
    ether.browseUrl(PSTR("/getrr.php?lat=52&lon=4"), NULL, website, my_callback);
  }
}

Unfortunately it seems that the website returns more data than my buffer can handle. The response is truncated somewhere in the middle as a result.

Is it possible to receive the contents "line by line"? I don't have to store the contents, I'm just interested in some datafields on each line....

2 
byte Ethernet::buffer[700];

// called when the client request is complete
static void my_callback (byte status, word off, word len) {
  Serial.println(">>>");
  Ethernet::buffer[off+300] = 0;   ////////// Perhaps dropping a NULL terminator at offset+300 is the problem.  Try offset+length.
  Serial.print((const char*) Ethernet::buffer + off);
  Serial.println("...");
}
3

Get the latest version of Ethercard from GitHub
, there is a new function
persistTcpConnection(bool persist)

Set it to true, and your callback will be called for each data packet that is recieved in response to you browseUrl call