State of Arduino Ethernet Library

I’m new to the Ethernet Library so please excuse if I make some wrong assumptions here. I’m just looking for a more generic way to do things.

  • Is there someone maintaining the Library? I could not find a name anywhere.
  • Looks like the whole library is only useful for W5100 chips, every file links to it.
  • Why not extract a generic interface which could support multiple implementations?
  • DHCP works nicely but what’s about DNS? The existing code does not work for me.

I’m asking because I ported this library to the W5200 on a Teensy3. Almost all code is identical to the Ethernet library but I had to replace this <<#include “w5100.h”>> in every single file.

For me it looks like more or less every Ethernet chipset should be usable through one generic interface… Is there someone interested in this? What about the original author(s) of the Ethernet Library?

cheers,
Thomas

The libraries and the related docs are maintained basically by us with the Arduino code crew editing the code, and the webmaster of the Arduino site doing the documentation editing. That webmaster person has been getting a bit behind lately it seems. It is a thankless job, so I'm not complaining, just saying...

I have submitted changes and bug fixes to the ethernet library, as have others here. It seems to be doing much better since IDE v1.0.1. :)

For me it looks like more or less every Ethernet chipset should be usable through one generic interface… Is there someone interested in this?

That is a GREAT idea! Contact Microchip and tell them to "get on the stick" and get their ENC28J60 hardware compatible with the w5100. ;)

What about the original author(s) of the Ethernet Library?

The original author(s) based the ethernet library around the w5100.h and w5100.cpp files provided by Wiznet. I have already submitted changes to the Arduino code crew that allows the ethernet library to use the w5200, the newer of the Wiznet products.

BTW, there are generic Client, Server, IPAddress classes in the core, that various implementations can inherit from to provide a common interface. It probably doesn’t share as much as it could but it’s something.

mellis: BTW, there are generic Client, Server, IPAddress classes in the core, that various implementations can inherit from to provide a common interface. It probably doesn't share as much as it could but it's something.

I can see how the libraries could be designed to use the same function calls, but different low level libraries. It would be great if you could compile for an ENC28J60 just by changing the include file. Is there enough reason to add that? Does the ENC28J60 offer advantages the W5200 does not?

This is just a transport, not a protocol. Setting up the transport should be rather universal. I mean, how much is there to it? mac, ip, subnet, gateway, dns, dhcp.

Thanks for all your help, David. Your efforts do not go unnoticed by at least some here. :) Karma +1 for you.

Thanks for your fast reply. Looks like we're thinking in the same direction.

SurferTim: This is just a transport, not a protocol. Setting up the transport should be rather universal. I mean, how much is there to it? mac, ip, subnet, gateway, dns, dhcp.

For me the transport-layer should provide stuff like: mac, ip, subnet, gateway, sockets (open, close, send, getFreeTxSpace, getBytesAvailable). I'm quite sure it's possible to abstract this for (at least) the most common Ethernet chips out there.

Currently all implemented protocols (EthernetUDP, EthernetClient, DNS and DHCP) do include the w5100.h which binds it to this specific chipset. Some people just replace the content of the w5100.h,.cpp files with a driver for the W5200 chip. I don't like this workaround at all...

I only had to replace two functions (and some minor adjustments) in W5100.cpp and it worked for a W5200 on a 32bit ARM Cortex-M4 CPU. This is just great! The different layers (transport, IP/UDP, highlevel-protocol using TCP/UDP) are already layered how it should be but the binding is too tight.

Actually I don't know much about inheritance, interfaces, linking and stuff like this in C++ as I do have a Java background. Shouldn't this be possible?

We are together here. I see where you are going, but it may require more than meets the eye. How would you tell the devices apart before compile? You must select the correct Arduino board in the IDE before compile. It does not even do that automatically.

I thought maybe using a different ethernet library include file for each. I wouldn’t want to include all the code for every ethernet device, just the one I am using.

#include <Ethernet.h>
// or
#include <Ethernet5200.h>
// or
#include <Ethernet28J60.h>

All could include the same functions with the same parameters and return values as the current ethernet library.
Ethernet.begin()
server.available()
client.read()
client.println()
client.stop()
…and the like, don’t you think?

@SurferTim, is there an online repository (github, Google Code, etc.) where an adventurous person could download the library with the mods and enhancement that have not yet made their way into the IDE?

@Jack Christensen: Here is the request I submitted on the code.google.com site. It has been imported into the github site now. http://code.google.com/p/arduino/issues/detail?id=898 https://github.com/arduino/Arduino/issues/898 It does the w5200 conversion part, but not the ENC28J60.

edit: Careful on the Ethernet.h and Ethernet.cpp files. Back up your current files first. They may have changed since I posted that code.

SurferTim: @Jack Christensen: Here is the request I submitted on the code.google.com site. It has been imported into the github site now.

Great, thanks! I've just recently started playing with the W5200 (on a WIZ810io module).

@Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

SurferTim: @Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

Sure will. Not sure if there's an ENC28J60 in my future though. Don't know much about it, but what little I've read makes me think it's a less capable chip, by which I just mean it does less in hardware, so the software has to pick up some functionality.

SurferTim:
We are together here. I see where you are going, but it may require more than meets the eye. How would you tell the devices apart before compile? You must select the correct Arduino board in the IDE before compile. It does not even do that automatically.

I thought maybe using a different ethernet library include file for each. I wouldn’t want to include all the code for every ethernet device, just the one I am using.

#include <Ethernet.h>

// or
#include <Ethernet5200.h>
// or
#include <Ethernet28J60.h>



All could include the same functions with the same parameters and return values as the current ethernet library. 
Ethernet.begin()
server.available()
client.read()
client.println()
client.stop()
...and the like, don't you think?

That’s what I’m looking at too. Actually I found a nice solution where there is one common, device-independent Ethernet class which is never used directly but only as interface. The actual implementation is EthernetW5200 which could easily be replaced with other implementations. What I like about my solution is that there is absolutely no change needed in the library files. The ‘include’ in the Sketch decides about the implementation. What I don’t like about this solution is the amount of work which has to be done before this’ll work. TCP and UDP implementation are tightly coupled to the W5x00 module.

W5100 and W5200 are almost identical, so it’s possible to do them in the same sourcecode files but I doubt it’s the same with ENC28J60 and others. IMHO this is only a short-term solution.

I’ve already uploaded some stuff to github too. It’s not much more than a proof-of-concept but it already shows that it’s possible to include different header files to select an implementation. Here’s what I did:

  • Made EthernetClass an abstract class which’ll be the only interface used by all protocol implementations.
  • Changed EthernetW5200 class so it inherits methods from the EthernetClass
  • The Sketch now has to include one implementation and assign it to the global variable ‘Ethernet’
  • All protocol implementations are unchanged but won’t work until they are freed from the direct calls to W5x00.*

And finally the link: GitHub - tht/ArduinoEthernetGeneric: Arduino Ethernet Library with support for different Ethernet Chipsets

I won’t be able to spend much time on it for a few weeks but it’s a start and shows a possible way to go.

@Tim, I downloaded the four files from Issue 898 on Google, Comment 6 by alexis.z...@gmail.com, Jun 14, 2012. Is that correct?

Those should be good.

I needed the Arduino Ethernet Library to give me back the DHCP-server’s IP address, So I changed the Ethernet.h and Ethernet.cpp files (Arduino IDE 1.0.3)

Now one can use this in his sketch to get the DHCP-server’s IP-address:

byte dhcpip[4];
for (byte i = 0; i < 4; i++)
{
  dhcpip[i] = Ethernet.dhcpServerIP()[i];
}

could someone submit these changes?

The topic I used to help me on the dhcp subject:
http://arduino.cc/forum/index.php/topic,133670.0

This is what I added in Ethernet.h (between the *********************************):

#ifndef ethernet_h
#define ethernet_h

#include <inttypes.h>
//#include "w5100.h"
#include "IPAddress.h"
#include "EthernetClient.h"
#include "EthernetServer.h"
#include "Dhcp.h"

#define MAX_SOCK_NUM 4

class EthernetClass {
private:
  IPAddress _dnsServerAddress;
//**************************************
  IPAddress _dhcpServerAddress;
//**************************************
  DhcpClass* _dhcp;
public:
  static uint8_t _state[MAX_SOCK_NUM];
  static uint16_t _server_port[MAX_SOCK_NUM];
  // Initialise the Ethernet shield to use the provided MAC address and gain the rest of the
  // configuration through DHCP.
  // Returns 0 if the DHCP configuration failed, and 1 if it succeeded
  int begin(uint8_t *mac_address);
  void begin(uint8_t *mac_address, IPAddress local_ip);
  void begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server);
  void begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server, IPAddress gateway);
  void begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server, IPAddress gateway, IPAddress subnet);
  int maintain();

  IPAddress localIP();
  IPAddress subnetMask();
  IPAddress gatewayIP();
  IPAddress dnsServerIP();
//*****************************************
  IPAddress dhcpServerIP();
//*****************************************
  friend class EthernetClient;
  friend class EthernetServer;
};

extern EthernetClass Ethernet;

#endif

This is what I added in Ethernet.cpp (between the *********************************):

#include "w5100.h"
#include "Ethernet.h"
#include "Dhcp.h"

// XXX: don't make assumptions about the value of MAX_SOCK_NUM.
uint8_t EthernetClass::_state[MAX_SOCK_NUM] = { 
  0, 0, 0, 0 };
uint16_t EthernetClass::_server_port[MAX_SOCK_NUM] = { 
  0, 0, 0, 0 };

int EthernetClass::begin(uint8_t *mac_address)
{
  _dhcp = new DhcpClass();


  // Initialise the basic info
  W5100.init();
  W5100.setMACAddress(mac_address);
  W5100.setIPAddress(IPAddress(0,0,0,0).raw_address());

  // Now try to get our config info from a DHCP server
  int ret = _dhcp->beginWithDHCP(mac_address);
  if(ret == 1)
  {
    // We've successfully found a DHCP server and got our configuration info, so set things
    // accordingly
    W5100.setIPAddress(_dhcp->getLocalIp().raw_address());
    W5100.setGatewayIp(_dhcp->getGatewayIp().raw_address());
    W5100.setSubnetMask(_dhcp->getSubnetMask().raw_address());
    _dnsServerAddress = _dhcp->getDnsServerIp();

   //**************************************
    _dhcpServerAddress = _dhcp->getDhcpServerIp();
   //**************************************
    
  }

  return ret;
}

void EthernetClass::begin(uint8_t *mac_address, IPAddress local_ip)
{
  // Assume the DNS server will be the machine on the same network as the local IP
  // but with last octet being '1'
  IPAddress dns_server = local_ip;
  dns_server[3] = 1;
  begin(mac_address, local_ip, dns_server);
}

void EthernetClass::begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server)
{
  // Assume the gateway will be the machine on the same network as the local IP
  // but with last octet being '1'
  IPAddress gateway = local_ip;
  gateway[3] = 1;
  begin(mac_address, local_ip, dns_server, gateway);
}

void EthernetClass::begin(uint8_t *mac_address, IPAddress local_ip, IPAddress dns_server, IPAddress gateway)
{
  IPAddress subnet(255, 255, 255, 0);
  begin(mac_address, local_ip, dns_server, gateway, subnet);
}

void EthernetClass::begin(uint8_t *mac, IPAddress local_ip, IPAddress dns_server, IPAddress gateway, IPAddress subnet)
{
  W5100.init();
  W5100.setMACAddress(mac);
  W5100.setIPAddress(local_ip._address);
  W5100.setGatewayIp(gateway._address);
  W5100.setSubnetMask(subnet._address);
  _dnsServerAddress = dns_server;
}

int EthernetClass::maintain(){
  int rc = DHCP_CHECK_NONE;
  if(_dhcp != NULL){
    //we have a pointer to dhcp, use it
    rc = _dhcp->checkLease();
    switch ( rc ){
      case DHCP_CHECK_NONE:
        //nothing done
        break;
      case DHCP_CHECK_RENEW_OK:
      case DHCP_CHECK_REBIND_OK:
        //we might have got a new IP.
        W5100.setIPAddress(_dhcp->getLocalIp().raw_address());
        W5100.setGatewayIp(_dhcp->getGatewayIp().raw_address());
        W5100.setSubnetMask(_dhcp->getSubnetMask().raw_address());
        _dnsServerAddress = _dhcp->getDnsServerIp();
        break;
      default:
        //this is actually a error, it will retry though
        break;
    }
  }
  return rc;
}

IPAddress EthernetClass::localIP()
{
  IPAddress ret;
  W5100.getIPAddress(ret.raw_address());
  return ret;
}

IPAddress EthernetClass::subnetMask()
{
  IPAddress ret;
  W5100.getSubnetMask(ret.raw_address());
  return ret;
}

IPAddress EthernetClass::gatewayIP()
{
  IPAddress ret;
  W5100.getGatewayIp(ret.raw_address());
  return ret;
}

IPAddress EthernetClass::dnsServerIP()
{
  return _dnsServerAddress;
}

//**************************************
IPAddress EthernetClass::dhcpServerIP()
{
  return _dhcpServerAddress;
}
//**************************************

EthernetClass Ethernet;

Ethernet.h (1.26 KB)

Ethernet.cpp (3.43 KB)

JO3RI: So I changed the Ethernet.h and Ethernet.cpp files (Arduino IDE 1.0.3)

How have you changed them? I mean, what have you changed in them?

Look for this in the code. The additions are between these:

//*************************************

There are two additions in each file. I have not checked the code changes for operation.

SurferTim:
@Jack: Let me know how it does for you. If good, then we only have one more to go. The ENC28J60.

@Tim, for a week or more now, I’ve been running a Cosm test sketch which posts six data points to each of five data feeds once per minute, with five seconds between posts, using a WIZ820io. The new library code seems to work as well as before. This is not to say without issues, but I suspect these are due to a combination of network or web site delays and perhaps my code :fearful:

Probably not perfectly definitive results, but I hope it helps.

Thanks, Jack. What are the issues? Does the code fail? I tested mine by creating issues, and insuring the condition did not crash the sketch. Three of my favorite issues to create:

  • Attempt to connect to a server that doesn't exist.
  • Attempt to get a dhcp ip address with no dhcp server.
  • Connect to a known large website and break the connection during the server response. Google home page is large, so I used that for my test.

That last one should work fine with the code I posted in the playground. It shows "Timeout" on the serial monitor when the transport connection fails, but goes back to downloading again when the transport connection is fixed. I broke the connection by unplugging my router CAT5 cable from the cablemodem.

SurferTim: Thanks, Jack. What are the issues? Does the code fail?

The code doesn't fail, but I have not been able to achieve the reliability that I would like in posting data to sites like Cosm and Thingspeak, meaning that sometimes the data just doesn't get there. My main application is logging data from wireless (XBee-based) sensor networks to these web sites. For instance, one network I have has five sensor units, each transmitting five or six data points once per minute. Each unit has its own time slot to transmit, and these are spaced five seconds apart. These all funnel to a central unit with the Ethernet interface.

The code does the usual thing, posts the data, waits for the server to respond and disconnect, then it disconnects. Sometimes the response doesn't arrive or the server doesn't disconnect before the next sensor's data arrives. Some amount of this is understandable due to network or server delays. If this happens, the code will disconnect, i.e. client.stop(), reset the WizNet module, and carry on. Sometimes the data gets posted on the web site OK even though a response didn't arrive (or didn't arrive quickly). I assume that could be network delays, etc. But sometimes it just doesn't get posted.

Initially I thought I had a reasonable aggregate data rate, but maybe not, maybe I'm pushing it too hard. I didn't want to get into queueing data, not enough memory for much of that, and while I fully expect that I may lose the occasional post with this approach, it's a matter of degree, I just haven't been able to get to the level of reliability that I'd like.

Sorry for the long story, don't expect you to debug my code necessarily, but if you have ideas I'd be glad to hear them too. I'll have a look at the code in the playground and see what I can learn from it. I very much appreciate your efforts for the community!