State of Arduino Ethernet Library

Thanks for your fast reply. Looks like we're thinking in the same direction.

SurferTim:
This is just a transport, not a protocol. Setting up the transport should be rather universal. I mean, how much is there to it? mac, ip, subnet, gateway, dns, dhcp.

For me the transport-layer should provide stuff like: mac, ip, subnet, gateway, sockets (open, close, send, getFreeTxSpace, getBytesAvailable). I'm quite sure it's possible to abstract this for (at least) the most common Ethernet chips out there.

Currently all implemented protocols (EthernetUDP, EthernetClient, DNS and DHCP) do include the w5100.h which binds it to this specific chipset. Some people just replace the content of the w5100.h,.cpp files with a driver for the W5200 chip. I don't like this workaround at all...

I only had to replace two functions (and some minor adjustments) in W5100.cpp and it worked for a W5200 on a 32bit ARM Cortex-M4 CPU. This is just great! The different layers (transport, IP/UDP, highlevel-protocol using TCP/UDP) are already layered how it should be but the binding is too tight.

Actually I don't know much about inheritance, interfaces, linking and stuff like this in C++ as I do have a Java background. Shouldn't this be possible?

We are together here. I see where you are going, but it may require more than meets the eye. How would you tell the devices apart before compile? You must select the correct Arduino board in the IDE before compile. It does not even do that automatically.

I thought maybe using a different ethernet library include file for each. I wouldn't want to include all the code for every ethernet device, just the one I am using.

#include <Ethernet.h>
// or
#include <Ethernet5200.h>
// or
#include <Ethernet28J60.h>

All could include the same functions with the same parameters and return values as the current ethernet library.
Ethernet.begin()
server.available()
client.read()
client.println()
client.stop()
...and the like, don't you think?

@SurferTim, is there an online repository (github, Google Code, etc.) where an adventurous person could download the library with the mods and enhancement that have not yet made their way into the IDE?

@Jack Christensen: Here is the request I submitted on the code.google.com site. It has been imported into the github site now.
Google Code Archive - Long-term storage for Google Code Project Hosting.
w5200 library code [imported] · Issue #898 · arduino/Arduino · GitHub
It does the w5200 conversion part, but not the ENC28J60.

edit: Careful on the Ethernet.h and Ethernet.cpp files. Back up your current files first. They may have changed since I posted that code.

SurferTim:
@Jack Christensen: Here is the request I submitted on the code.google.com site. It has been imported into the github site now.

Great, thanks! I've just recently started playing with the W5200 (on a WIZ810io module).

@Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

SurferTim:
@Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

Sure will. Not sure if there's an ENC28J60 in my future though. Don't know much about it, but what little I've read makes me think it's a less capable chip, by which I just mean it does less in hardware, so the software has to pick up some functionality.

SurferTim:
We are together here. I see where you are going, but it may require more than meets the eye. How would you tell the devices apart before compile? You must select the correct Arduino board in the IDE before compile. It does not even do that automatically.

I thought maybe using a different ethernet library include file for each. I wouldn't want to include all the code for every ethernet device, just the one I am using.

#include <Ethernet.h>

// or
#include <Ethernet5200.h>
// or
#include <Ethernet28J60.h>



All could include the same functions with the same parameters and return values as the current ethernet library. 
Ethernet.begin()
server.available()
client.read()
client.println()
client.stop()
...and the like, don't you think?

That's what I'm looking at too. Actually I found a nice solution where there is one common, device-independent Ethernet class which is never used directly but only as interface. The actual implementation is EthernetW5200 which could easily be replaced with other implementations. What I like about my solution is that there is absolutely no change needed in the library files. The 'include' in the Sketch decides about the implementation. What I don't like about this solution is the amount of work which has to be done before this'll work. TCP and UDP implementation are tightly coupled to the W5x00 module.

W5100 and W5200 are almost identical, so it's possible to do them in the same sourcecode files but I doubt it's the same with ENC28J60 and others. IMHO this is only a short-term solution.

I've already uploaded some stuff to github too. It's not much more than a proof-of-concept but it already shows that it's possible to include different header files to select an implementation. Here's what I did:

  • Made EthernetClass an abstract class which'll be the only interface used by all protocol implementations.
  • Changed EthernetW5200 class so it inherits methods from the EthernetClass
  • The Sketch now has to include one implementation and assign it to the global variable 'Ethernet'
  • All protocol implementations are unchanged but won't work until they are freed from the direct calls to W5x00.*

And finally the link: GitHub - tht/ArduinoEthernetGeneric: Arduino Ethernet Library with support for different Ethernet Chipsets

I won't be able to spend much time on it for a few weeks but it's a start and shows a possible way to go.

@Tim, I downloaded the four files from Issue 898 on Google, Comment 6 by alexis.z...@gmail.com, Jun 14, 2012. Is that correct?

Those should be good.

I needed the Arduino Ethernet Library to give me back the DHCP-server's IP address, So I changed the Ethernet.h and Ethernet.cpp files (Arduino IDE 1.0.3)

Now one can use this in his sketch to get the DHCP-server's IP-address:

byte dhcpip[4];
for (byte i = 0; i < 4; i++)
{
  dhcpip[i] = Ethernet.dhcpServerIP()[i];
}

could someone submit these changes?

The topic I used to help me on the dhcp subject:
http://arduino.cc/forum/index.php/topic,133670.0

This is what I added in Ethernet.h (between the *********************************):

#ifndef ethernet_h
#define ethernet_h

#include <inttypes.h>
//#include "w5100.h"
#include "IPAddress.h"
#include "EthernetClient.h"
#include "EthernetServer.h"
#include "Dhcp.h"

#define MAX_SOCK_NUM 4

class EthernetClass {
private:
  IPAddress _dnsServerAddress;
//**************************************
  IPAddress _dhcpServerAddress;
//**************************************
  DhcpClass* _dhcp;
public:
  static uint8_t _state[MAX_SOCK_NUM];
  static uint16_t _server_port[MAX_SOCK_NUM];
  // Initialise the Ethernet shield to use the provided MAC address and gain the rest of the
  // configuration through DHCP.
  // Returns 0 if the DHCP configuration failed, and 1 if it succeeded
  int begin(uint8_t *mac_address);
  void begin(uint8_t *mac_address, IPAddress local_ip);
  void begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server);
  void begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server, IPAddress gateway);
  void begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server, IPAddress gateway, IPAddress subnet);
  int maintain();

  IPAddress localIP();
  IPAddress subnetMask();
  IPAddress gatewayIP();
  IPAddress dnsServerIP();
//*****************************************
  IPAddress dhcpServerIP();
//*****************************************
  friend class EthernetClient;
  friend class EthernetServer;
};

extern EthernetClass Ethernet;

#endif

This is what I added in Ethernet.cpp (between the *********************************):

#include "w5100.h"
#include "Ethernet.h"
#include "Dhcp.h"

// XXX: don't make assumptions about the value of MAX_SOCK_NUM.
uint8_t EthernetClass::_state[MAX_SOCK_NUM] = { 
  0, 0, 0, 0 };
uint16_t EthernetClass::_server_port[MAX_SOCK_NUM] = { 
  0, 0, 0, 0 };

int EthernetClass::begin(uint8_t *mac_address)
{
  _dhcp = new DhcpClass();


  // Initialise the basic info
  W5100.init();
  W5100.setMACAddress(mac_address);
  W5100.setIPAddress(IPAddress(0,0,0,0).raw_address());

  // Now try to get our config info from a DHCP server
  int ret = _dhcp->beginWithDHCP(mac_address);
  if(ret == 1)
  {
    // We've successfully found a DHCP server and got our configuration info, so set things
    // accordingly
    W5100.setIPAddress(_dhcp->getLocalIp().raw_address());
    W5100.setGatewayIp(_dhcp->getGatewayIp().raw_address());
    W5100.setSubnetMask(_dhcp->getSubnetMask().raw_address());
    _dnsServerAddress = _dhcp->getDnsServerIp();

   //**************************************
    _dhcpServerAddress = _dhcp->getDhcpServerIp();
   //**************************************
    
  }

  return ret;
}

void EthernetClass::begin(uint8_t *mac_address, IPAddress local_ip)
{
  // Assume the DNS server will be the machine on the same network as the local IP
  // but with last octet being '1'
  IPAddress dns_server = local_ip;
  dns_server[3] = 1;
  begin(mac_address, local_ip, dns_server);
}

void EthernetClass::begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server)
{
  // Assume the gateway will be the machine on the same network as the local IP
  // but with last octet being '1'
  IPAddress gateway = local_ip;
  gateway[3] = 1;
  begin(mac_address, local_ip, dns_server, gateway);
}

void EthernetClass::begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server, IPAddress gateway)
{
  IPAddress subnet(255, 255, 255, 0);
  begin(mac_address, local_ip, dns_server, gateway, subnet);
}

void EthernetClass::begin(uint8_t *mac, IPAddress local_ip, IPAddress dns_server, IPAddress gateway, IPAddress subnet)
{
  W5100.init();
  W5100.setMACAddress(mac);
  W5100.setIPAddress(local_ip._address);
  W5100.setGatewayIp(gateway._address);
  W5100.setSubnetMask(subnet._address);
  _dnsServerAddress = dns_server;
}

int EthernetClass::maintain(){
  int rc = DHCP_CHECK_NONE;
  if(_dhcp != NULL){
    //we have a pointer to dhcp, use it
    rc = _dhcp->checkLease();
    switch ( rc ){
      case DHCP_CHECK_NONE:
        //nothing done
        break;
      case DHCP_CHECK_RENEW_OK:
      case DHCP_CHECK_REBIND_OK:
        //we might have got a new IP.
        W5100.setIPAddress(_dhcp->getLocalIp().raw_address());
        W5100.setGatewayIp(_dhcp->getGatewayIp().raw_address());
        W5100.setSubnetMask(_dhcp->getSubnetMask().raw_address());
        _dnsServerAddress = _dhcp->getDnsServerIp();
        break;
      default:
        //this is actually a error, it will retry though
        break;
    }
  }
  return rc;
}

IPAddress EthernetClass::localIP()
{
  IPAddress ret;
  W5100.getIPAddress(ret.raw_address());
  return ret;
}

IPAddress EthernetClass::subnetMask()
{
  IPAddress ret;
  W5100.getSubnetMask(ret.raw_address());
  return ret;
}

IPAddress EthernetClass::gatewayIP()
{
  IPAddress ret;
  W5100.getGatewayIp(ret.raw_address());
  return ret;
}

IPAddress EthernetClass::dnsServerIP()
{
  return _dnsServerAddress;
}

//**************************************
IPAddress EthernetClass::dhcpServerIP()
{
  return _dhcpServerAddress;
}
//**************************************

EthernetClass Ethernet;

Ethernet.h (1.26 KB)

Ethernet.cpp (3.43 KB)

JO3RI:
So I changed the Ethernet.h and Ethernet.cpp files (Arduino IDE 1.0.3)

How have you changed them? I mean, what have you changed in them?

Look for this in the code. The additions are between these:

//*************************************

There are two additions in each file. I have not checked the code changes for operation.

SurferTim:
@Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

@Tim, for a week or more now, I've been running a Cosm test sketch which posts six data points to each of five data feeds once per minute, with five seconds between posts, using a WIZ820io. The new library code seems to work as well as before. This is not to say without issues, but I suspect these are due to a combination of network or web site delays and perhaps my code :fearful:

Probably not perfectly definitive results, but I hope it helps.

Thanks, Jack. What are the issues? Does the code fail? I tested mine by creating issues, and insuring the condition did not crash the sketch. Three of my favorite issues to create:

  • Attempt to connect to a server that doesn't exist.
  • Attempt to get a dhcp ip address with no dhcp server.
  • Connect to a known large website and break the connection during the server response. Google home page is large, so I used that for my test.

That last one should work fine with the code I posted in the playground. It shows "Timeout" on the serial monitor when the transport connection fails, but goes back to downloading again when the transport connection is fixed. I broke the connection by unplugging my router CAT5 cable from the cablemodem.

SurferTim:
Thanks, Jack. What are the issues? Does the code fail?

The code doesn't fail, but I have not been able to achieve the reliability that I would like in posting data to sites like Cosm and Thingspeak, meaning that sometimes the data just doesn't get there. My main application is logging data from wireless (XBee-based) sensor networks to these web sites. For instance, one network I have has five sensor units, each transmitting five or six data points once per minute. Each unit has its own time slot to transmit, and these are spaced five seconds apart. These all funnel to a central unit with the Ethernet interface.

The code does the usual thing, posts the data, waits for the server to respond and disconnect, then it disconnects. Sometimes the response doesn't arrive or the server doesn't disconnect before the next sensor's data arrives. Some amount of this is understandable due to network or server delays. If this happens, the code will disconnect, i.e. client.stop(), reset the WizNet module, and carry on. Sometimes the data gets posted on the web site OK even though a response didn't arrive (or didn't arrive quickly). I assume that could be network delays, etc. But sometimes it just doesn't get posted.

Initially I thought I had a reasonable aggregate data rate, but maybe not, maybe I'm pushing it too hard. I didn't want to get into queueing data, not enough memory for much of that, and while I fully expect that I may lose the occasional post with this approach, it's a matter of degree, I just haven't been able to get to the level of reliability that I'd like.

Sorry for the long story, don't expect you to debug my code necessarily, but if you have ideas I'd be glad to hear them too. I'll have a look at the code in the playground and see what I can learn from it. I very much appreciate your efforts for the community!

Thanks for the input. Here is a link to that client code.
http://playground.arduino.cc/Code/WebClient

I forgot my best issue to create. I have a network available with over 300 clients on it, so it gets a bit busy. That generates delays with my code, but not a failure. Unless the server is localnet, and you control that server and localnet, there will be occasional delays. BTW, I have a couple of those localnet servers. One on a busy network, and one on my personal network.

SurferTim:
@Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

is there an updated library for ENC28J60? actually im using this module ryt now.. i just want to run a W5100 based sketch on this module.. is there a way?

@pisayjames: AFAIK, there is not a library that will work with the ENC28J60 using the w5100 code. It seems like it should be possible by changing the include file, but nobody has developed one. At this time, I have only the w5100 shield.