Read webpage content with ENC28J60 - Help

Projects General Guidance
1

Hi,

I'm trying to get a project working where I pull variables from a webpage and then display them on the serial port (for now). I have your standard UNO Rev3 board, but a ENC28J60 instead of a Ethernet/Wifi Shield to keep initial cost down. I've run through examples on getting the board connect to the internet, however, I'm having trouble getting finding code to help me understand how to correctly read content from a webpage. Any links or advice to get me in the right direction are greatly appreciated.

My other concern is that since I'm using a ENC28J60 instead of the standard ethernet shields, is my code going to have to be completely different due to different libraries?

Lastly, I want my arduino to be able to access the script from any network... so on your standard DHCP server. Will the ENC28J60 be able to support this or will I have to host it on a static IP address?

Thanks in advance.

2

I'm having trouble finding your code. What web site are you trying to read?

My other concern is that since I'm using a ENC28J60 instead of the standard ethernet shields, is my code going to have to be completely different due to different libraries?

Of course it is.

Lastly, I want my arduino to be able to access the script from any network... so on your standard DHCP server. Will the ENC28J60 be able to support this or will I have to host it on a static IP address?

Don't know. But, at this point, I'm wondering if saving a few pennies was cost effective.

3

Some libraries support DHCP for the ENC28J60.
Which library do you use ?

4

Caltoa:
Some libraries support DHCP for the ENC28J60.
Which library do you use ?

I used an EtherCard library and at least got a connection to work. I tried using the domain with my script on it, and was able to get the server IP. However, when accessing the content, I got a 301 error I believe (don't have my board with me now). However, when I changed the website to another test domain I have with a static IP, it was able to connect and begin reading the content without issues.

Don't know. But, at this point, I'm wondering if saving a few pennies was cost effective.

I'm starting to think you are correct. I may just go pick up an Ethernet shield to save some future headache. But with that being said, would the code be drastically different if I eventually were to switch to a Wifi shield?

5

This was the code I was using

// This demo does web requests via DNS lookup, using a fixed gateway.
// 2010-11-27 <jc@wippler.nl> http://opensource.org/licenses/mit-license.php

#include <EtherCard.h>

#define REQUEST_RATE 5000 // milliseconds

// ethernet interface mac address
static byte mymac[] = { 0x74,0x69,0x69,0x2D,0x30,0x31 };
// ethernet interface ip address
static byte myip[] = { 192,168,1,203 };
// gateway ip address
static byte gwip[] = { 192,168,1,1 };
// remote website name
char website[] PROGMEM = "domainnameexample.com";

byte Ethernet::buffer[300];   // a very small tcp/ip buffer is enough here
static long timer;

// called when the client request is complete
static void my_result_cb (byte status, word off, word len) {
  Serial.print("<<< reply ");
  Serial.print(millis() - timer);
  Serial.println(" ms");
  Serial.println((const char*) Ethernet::buffer + off);
}

void setup () {
  Serial.begin(57600);
  Serial.println("\n[getViaDNS]");
  
  if (ether.begin(sizeof Ethernet::buffer, mymac) == 0) 
    Serial.println( "Failed to access Ethernet controller");

  ether.staticSetup(myip, gwip);

  if (!ether.dnsLookup(website))
    Serial.println("DNS failed");
  ether.printIp("Server: ", ether.hisip);
  
  timer = - REQUEST_RATE; // start timing out right away
}

void loop () {
  ether.packetLoop(ether.packetReceive());
  
  if (millis() > timer + REQUEST_RATE) {
    timer = millis();
    Serial.println("\n>>> REQ");
    ether.browseUrl(PSTR("/test/script.php"), "", website, my_result_cb);
  }
}
6

Joffroi:
My other concern is that since I'm using a ENC28J60 instead of the standard ethernet shields, is my code going to have to be completely different due to different libraries?

That depends on the library you use. If you choose UIPEthernet your code will be pretty much the same as with standard Ethernet-lib for WIZ5100 based shields

Joffroi:
Lastly, I want my arduino to be able to access the script from any network... so on your standard DHCP server. Will the ENC28J60 be able to support this or will I have to host it on a static IP address?

DHCP works fine with UIPEthernet. Only problem is it requires aprox. 5kb more flash than tcp with static-ip.

  • Norbert
7

Thanks for the links Norbert.

I was looking through the examples provided. Should I be able to use the structure of the TCPClient along with some of the function (server call and server, 80 connect and GET function from the standard webclient tutorial?

8

Also the Ethercard DHCP is working fine. There is an example for DHCP with that library.

I would be interested in reading a webpage myself, and extracting valuable data. It is hard to read a webpage line by line, since there could be no lines. I don't know a good example yet.

9

Joffroi:
Should I be able to use the structure of the TCPClient along with some of the function (server call and server, 80 connect and GET function from the standard webclient tutorial?

yes, you may even just take the WebClientRepeating sketch and replace the include of Ethernet.h with UIPEthernet.h. Be aware there's a but in WebClientRepeating-sketch not using if (client.connect(server, 80) > 0) as Client.connect() may return negative values in case dns-lookup fails.

10

I ended up getting the arduino WiFi shield and got the project completely working. I was able to parse data from a php page and then display it to adafruit 7 segment displays. Thanks for the input!