neteemrom confusion

Guys,
The neteeprom library says this:

#define MAC_OFFSET (eepromOffset + 1)
#define DHCP_OFFSET (MAC_OFFSET + 6)
#define IP_OFFSET (DHCP_OFFSET + 1)
#define DNS_OFFSET (IP_OFFSET + 4)
#define GW_OFFSET (DNS_OFFSET + 4)
#define SUBNET_OFFSET (GW_OFFSET + 4)

Now, I'm a bit confused... eeprom offset + 1 why for MAC?? And then DHCP then why MAC OFFSET + 6??

aq_mishu:
eeprom offset + 1 why for MAC??

The library has a function NetEeprom.setEepromOffset() that allows you to offset the address where the network configuration is stored in EEPROM. Let's say your code was already using EEPROM addresses 0-15 for something else. The NetEeprom library uses an eepromOffset value of 0 so the MAC would be written to EEPROM address 1, the DHCP flag will be written to EEPROM address 7, the IP address will be written to EEPROM address 8, etc. So that will conflict with the EEPROM addresses 0-15 you're already using in your code. Rather than having to adjust your code to avoid the EEPROM addresses 1-24 that the NetEEPROM library uses by default, you could just use NetEeprom.setEepromOffset(15), which will cause the library to instead use EEPROM addresses 16-39.

So why does the library avoid using EEPROM address 0? Maybe the answer is found here:

In older generation AVRs the EEPROM Address Register (EEAR) is initialized to zero on reset, be it from Brown Out Detect, Watchdog or the Reset Pin. If an EEPROM write has just started at the time of the reset, the write will be completed, but now at address zero instead of the requested address. If the reset occurs later in the write process both the requested address and address zero may be corrupted.

aq_mishu:
MAC OFFSET + 6??

Let's say you are using the eepromOffset value of 0 so MAC_OFFSET is 1. A MAC is something like this:

byte mac[6] = { 0xCA, 0xFE, 0xBA, 0xBE, 0x00, 0x00 };

That's a 6 byte array. You can't fit that in a single EEPROM address, which can only hold a byte. Since it's a 6 byte array it takes 6 bytes of EEPROM, starting at MAC_OFFSET. So you need to write the DHCP flag to MAC_OFFSET + 6 so that you don't overwrite the addresses used for the MAC.

hi,
Thanks for the tutorial... so helpful and informative...

So, eeprom offset + 1 means MAC will actually be starting from address 1 (offset = 0 initially if not defined) as leaving address 0 is just for in case as a fail safe...

If i;m not wrong...

Also another thing, though aprt from this topic...

my previous code was:

// interface initialization
void inetInit() {
  if(usefixedip) {
    Ethernet.begin(mac, ip, dnsServer, gateway, subnet);
    inetOK = true;
    delay(1000);
    Serial.print("Local IP: ");
    Serial.println(Ethernet.localIP());
    Serial.println("Ethernet UP and Ready...");
  } 
  else {
    if (Ethernet.begin(mac) == 0) {
      inetOK = false;
      Serial.println("Ethernet filed to up...");
    } 
    else {
      inetOK = true;
    }
  }
  delay(1000);
}

And this was a init function that i used to call when needed. Now, i am planning to use the neteeprom and thus, here i the new code:

void inetInit() {
  NetEeprom.begin();
  delay(1000);
  inetOK = true;
  if (NetEeprom.checkMagic()) {
    byte mac[6];
    NetEeprom.readMac(mac);
    Serial.print("MAC: "); printMac(mac); Serial.println();
    if (NetEeprom.isDhcp()) {
      Serial.println("Network configured using DHCP");
    } else {
      byte addr[4];
      NetEeprom.readIp(addr);
      Serial.print("IP: "); printIp(addr); Serial.println();
      NetEeprom.readDns(addr);
      Serial.print("DNS: "); printIp(addr); Serial.println();
      NetEeprom.readGateway(addr);
      Serial.print("GW: "); printIp(addr); Serial.println();
      NetEeprom.readSubnet(addr);
      Serial.print("Subnet: "); printIp(addr); Serial.println();
    }
  } else {
    Serial.println("Network MAC and IP have not been configured");
  }
  delay(1000);
  Serial.println("Ethernet is up.");
}

The problem is I need to set the variable "inetOK = true;" or false like

if (Ethernet.begin(mac) == 0) {
      inetOK = false;
      Serial.println("Ethernet filed to up...");
    }

how can i get this in neteeprom? ideas??

aq_mishu:
So, eeprom offset + 1 means MAC will actually be starting from address 1 (offset = 0 initially if not defined) as leaving address 0 is just for in case as a fail safe...

correct

aq_mishu:
how can i get this in neteeprom? ideas??

NetEEPROM only give you a way to store and retrieve network configuration information in EEPROM. You still need to call Ethernet.begin(mac) so that code can remain unchanged.

aq_mishu:
So, eeprom offset + 1 means MAC will actually be starting from address 1 (offset = 0 initially if not defined) as leaving address 0 is just for in case as a fail safe...

I was actually wrong about that. It turns out that the EEPROM address eepromOffset is used to store a "magic number". When you call NetEEPROM.writeMac() the magic number is also written to that address. You can use NetEEPROM.checkMagic() to check if that address contains the magic number. If it returns false then you know that the network configuration has not yet been done. The library also checks if that address contains the magic number when NetEEPROM.begin() is called and if not it stores a randomly generated MAC in EEPROM.

pert:
correct
NetEEPROM only give you a way to store and retrieve network configuration information in EEPROM. You still need to call Ethernet.begin(mac) so that code can remain unchanged.

but it seems NetEEPROM.begin() is already doing that. And as I need to make a init confirmation, I can change the library from void to int and return 1 so I can check by if ( NetEEPROM.begin() != 1) { isConnected = false; }

If i'm not wrong.

OK, so about the magic of the library, may be... coz leaving a bit to a known place means there is a config available... not bad idea at all...

I'm planning to modify it, so it will not save all of the things, rather it will save WHAT I NEED. I will come to that part later...

please tell me another thing, I loaded the sketch in to uno and took out the eth shield away from it. thus practically it does not have any NIC. But i have seen it is showing "connected"...

Is there any way I can really understand if it is connected (or aka, if ethernet is really UP? [except ping])

aq_mishu:
but it seems NetEEPROM.begin() is already doing that.

You're right. I hadn't realized that. It's really silly because they shouldn't assume people want to use the Ethernet library. I have never used the library. I was aware of it because it caused a conflict with a different library of the same name that is part of a bootloader package that I have done some work on. At that time I did look at it close enough to find a bug and report it but that was some time ago.

aq_mishu:
And as I need to make a init confirmation, I can change the library from void to int and return 1 so I can check by if ( NetEEPROM.begin() != 1) { isConnected = false; }

You would want to change it so NetEEPROM.begin() has a bool return type and then return the result of Ethernet.begin():

return Ethernet.begin(mac, ip, dns, gw, subnet);

It's also easy enough to work around this without modifying the library. You would just reproduce the code of NetEEPROM.begin() in your sketch. So you are using readIp(ip), readDns(dns), readGateway(gw), and readSubnet(subnet) as needed and then passing those values to Ethernet.begin(), then determining the success from the return value.

pert:
readIp(ip), readDns(dns), readGateway(gw), and readSubnet(subnet) as needed and then passing those values to Ethernet.begin(), then determining the success from the return value.

parhaps... may be... time to make a trial... not a bad idea at all... correcting it ONLY for eeprom part and using regular ethernet library is not a bad idea.

Hi, just need an expert advise in a comparison wise...

int inetInit() {
  inetOK = false;
  NetEeprom.begin();
  delay(2000);
  inetOK = true;
  return 1;
}

vs

int inetInit() {
  inetOK = false;
  byte mac[6];
  if (NetEeprom.checkMagic()) {
    NetEeprom.readMac(mac);
  } else {
    //NetEeprom.generateRandomMac(mac);
    //NetEeprom.writeDhcpConfig(mac);
  }
  if (NetEeprom.isDhcp()) {
    Ethernet.begin(mac);
    delay(2000);
    inetOK = true;
    return 1;
  }
  else {
    byte ip[4];
    byte dns[4];
    byte gw[4];
    byte subnet[4];

    NetEeprom.readIp(ip);
    NetEeprom.readDns(dns);
    NetEeprom.readGateway(gw);
    NetEeprom.readSubnet(subnet);

    Ethernet.begin(mac, ip, dns, gw, subnet);
    delay(2000);
    inetOK = true;
    return 1;
  }
}

the fist one says:

avrdude: verifying ...
avrdude: 21008 bytes of flash verified

and the second one said:

avrdude: verifying ...
avrdude: 20788 bytes of flash verified

And another note: The Neteeprom library has not been changed at all. Also I dont care abut generating random MAC, though I do care about DHCP.

Now any expert opinions and comparisons??

Mishu~

Update:

  • EEPROM MAP, each address has 1 byte storage.
    • DEFAULT eepromOffset = 0 [Address 0]
  • #define MAC_OFFSET (eepromOffset + 1) [Start at 0+1=1, used for MAC (6byte), ends at 6]
  • #define DHCP_OFFSET (MAC_OFFSET + 6) [Start at 1+6=7, used for DHCP config (1byte), ends at 7]
  • #define IP_OFFSET (DHCP_OFFSET + 1) [Start at 7+1=8, used for IP config (4byte), ends at 11]
  • #define DNS_OFFSET (IP_OFFSET + 4) [Start at 11+4=15, used for DNS config (4byte), ends at 18]
  • #define GW_OFFSET (DNS_OFFSET + 4) [Start at 18+4=22, used for Gateway config (4byte), ends at 25]
  • #define SUBNET_OFFSET (GW_OFFSET + 4) [Start at 25+4=29, used for Subnet config (4byte), ends at 32]
  • Anything else is to be used from Address 33 and onwards. [Aka. ADDR 0-32 RESERVED for NETWORK CONFIG]

is this okey?

Hi,
another issue,
as per your advise, I tired to modify the library neteeeprom and no luck

Here is the function i made by editing the thing:

NetEeprom.writeIPConfig(ip);

void NetEEPROM::writeIPConfig(byte ip[]) {
  writeEEPROM(ip, IP_OFFSET, 4);
}

and also added is in header file. But then it's still not compiling...

aq_mishu:
I dont care abut generating random MAC

Then you can change this code:

  if (NetEeprom.checkMagic()) {
    NetEeprom.readMac(mac);
  } else {
    //NetEeprom.generateRandomMac(mac);
    //NetEeprom.writeDhcpConfig(mac);
  }

to:

NetEeprom.readMac(mac);

The important thing with MAC is to make sure it's unique on your network. If you haven't already written a MAC to EEPROM before calling Ethernet.begin() then it will just end up using whatever values happen to be in those EEPROM addresses. Usually the EEPROM is erased when you do a Burn Bootloader so this would cause the MAC to default to FF:FF:FF:FF:FF:FF (255:255:255:255:255:255 in decimal notation). If you had multiple devices on your network using this MAC address they would conflict and you would have problems with one of them. In this respect, your current code is perhaps better in that if a MAC has not been set then your mac array will be uninitialized and will contain whatever happens to be in that memory, which will cause it to be somewhat randomized. However, the call to NetEeprom.checkMagic() will add some memory overhead as well as the time required for checking the EEPROM for the magic number. Since you said you don't care about randomly generated MAC I assume your system is such that duplicate MAC addresses are not a serious concern.

aq_mishu:
as per your advise, I tired to modify the library neteeeprom and no luck

Actually I had advised against modifying the library. That was regarding your need for the return value of Ethernet.begin() and I explained that there is a simple workaround for that issue by only using the library to read and write network configuration to the EEPROM (which is all I originally assumed it did).

Adding the writeIPConfig function is a different subject matter and it does seem like a reasonable modification to make.

aq_mishu:
and also added is in header file. But then it's still not compiling...

Please post the full error message.

void NetEEPROM::writeIPConfig(byte ip[]) {
  writeEEPROM(ip, IP_OFFSET, 4);
}

in cpp file

void writeIPConfig(byte ip[]);

in .h file

and that perfectly worked... I forgot to add the ";" in the .h file... what an idiot i am...

By the way, about the random MAC, do you think that will be purely random? I mean there is still a chance of duplicate MACs right??

Now there is a new topic, apart from this thread but more of a related. I wish to store the server IP/URL in eeprom. Just storing strings. There are plenty of the methods i found in google, but noe actually was that much of a helpful. I have taken the approach as following:

if (str_rx.startsWith("set server ")) { //Set Server Address where it will knock)
      sub_str = str_rx.substring(11);
      sub_str.trim();
      Serial.println(sub_str);
      unsigned int strSize = sub_str.length(); //size calculated excluding NULL character '\0'
      EEPROM.write(SERVER_ADDR_SIZE, strSize); //has written 14 for www.mishu.com
      char strBuff[strSize]; //[13] for www.mishu.com
      sub_str.toCharArray(strBuff, strSize); //convert the string to an array of chars.

      Serial.println(sub_str.length());
      Serial.println(strSize);
      Serial.println(SERVER_ADDR_OFFSET);
      Serial.println(SERVER_ADDR_OFFSET+strSize);
      Serial.println();

      for (unsigned int i = SERVER_ADDR_OFFSET; i < (SERVER_ADDR_OFFSET+strSize); i++) {
        EEPROM.write(i, strBuff[i]);
      }
      
      EEPROM.write((SERVER_ADDR_OFFSET+strSize),'\0');  // Save the NULL terminator

      for (unsigned int i = SERVER_ADDR_OFFSET; i <= (SERVER_ADDR_OFFSET+strSize); i++) {
        //strBuff[i] = EEPROM.read(i);
        sub_str += EEPROM.read(i);
      }
      //sub_str = String(strBuff);
      Serial.println(sub_str);
      
      Serial.print(">");
      str_rx = "";
      sub_str = "";
    }

I write in serial consol as "set server www.mishu.com" and then it is supposed to store the "www.mishu.com" in the eeprom.

The code above is giving the below output...

>www.mishu.com
13
13
35
48

www.mishu.com219404001172000381390
>

Update::

Changed the code as per this:

if (str_rx.startsWith("set server ")) { //Set Server Address where it will knock)
      sub_str = str_rx.substring(11);
      sub_str.trim();
      Serial.println(sub_str);
      unsigned int strSize = sub_str.length()+1; //size calculated inccluding NULL character '\0'
      EEPROM.write(SERVER_ADDR_SIZE, strSize); //has written 14 for www.mishu.com
      char strBuff[strSize]; //[14] for www.mishu.com
      sub_str.toCharArray(strBuff, strSize); //convert the string to an array of chars.

      Serial.println(sub_str.length());
      Serial.println(strSize);
      Serial.println(SERVER_ADDR_OFFSET);
      Serial.println(SERVER_ADDR_OFFSET+strSize);
      Serial.println();

      for (unsigned int i = SERVER_ADDR_OFFSET; i < (SERVER_ADDR_OFFSET+strSize); i++) {
        EEPROM.write(i, strBuff[i]);
      }
      
      EEPROM.write((SERVER_ADDR_OFFSET+strSize),'\0');  // Save the NULL terminator

      sub_str = "";
      for (unsigned int i = SERVER_ADDR_OFFSET; i <= (SERVER_ADDR_OFFSET+strSize); i++) {
        strBuff[i] = EEPROM.read(i);
        //sub_str += EEPROM.read(i);
      }
      sub_str = String(strBuff);
      Serial.println(sub_str);
      Serial.println(sub_str.length());
      
      Serial.print(">");
      str_rx = "";
      sub_str = "";
    }

And got the output almost exactly I need...

>www.mishu.com
13
14
35
49

www.mishu.com
13

Which is fine, as I can see original string was 13 in size and after reading the string from eeprom, it's again 13. But I have added 13+1 as I need to consider the Null character. And even after considering NULL, the string should take address upto 48 (starting from ADDR 35 inclusive, the whole string takes till ADDR47 and with NULL 48). But the problem is still I can see it is taking 49 (35+14).

Actually as I have a plan of allocation of max string length by byte, if it is somehow reaches to max, it will loose 1 byte or overlap to next.

The way I see it is, 48 means 48... you can't go to 49. Any idea here??

Mishu~

Okkeyyyyyy Here it is... solved!! People may need it, and thus I'm giving it here...

if (str_rx.startsWith("set server ")) { //Set Server Address where it will knock)
      sub_str = str_rx.substring(11);
      sub_str.trim(); // example, www.banglardamal.org with a length = 20
      unsigned int strSize = sub_str.length()+1; //size calculated including NULL terminator '\0' We need it later for reading and last byte address known.
      EEPROM.write(SERVER_ADDR_SIZE, strSize); //has written 21 for www.banglardamal.org including NULL terminator.
      char strBuff[strSize]; //[21] for www.banglardamal.org
      sub_str.toCharArray(strBuff, strSize); //convert the string to an array of chars.

      //Debug purpose
      Serial.println(sub_str);
      Serial.println(sub_str.length());
      Serial.println(strSize);
      Serial.println(SERVER_ADDR_OFFSET);
      Serial.println(SERVER_ADDR_OFFSET+sub_str.length());
      Serial.println();
      //Debug purpose ends here

      for (unsigned int i = SERVER_ADDR_OFFSET; i < (SERVER_ADDR_OFFSET+sub_str.length()); i++) {
        EEPROM.write(i, strBuff[i]);
        Serial.print(i); //for debug
        Serial.print(";"); //for debug
      }

      Serial.println();
      EEPROM.write((SERVER_ADDR_OFFSET+sub_str.length()),'\0');  // Save the NULL terminator

      sub_str = "";
      for (unsigned int i = SERVER_ADDR_OFFSET; i < (SERVER_ADDR_OFFSET+strSize); i++) {
        strBuff[i] = EEPROM.read(i);
      }
      sub_str = String(strBuff);
      Serial.println(sub_str);
      Serial.println(sub_str.length()); //Debug purpose
      
      Serial.print(">");
      sub_str = "";
      str_rx = "";
    }

Run the code and you guys will see... Yes, it can be made more better and I hope someone will do that too, like making a function for the whole thing. So we can just pass parameters and it will do that for us. Also another improvement is left here... As we are using EEprom address in #define, we need to fix the max length (unless variable length all the way which needs dynamic memory allocation) and that I have no plan to discuss here in this thread. If I keep 100 bytes reserved, all I need a modification (update) in the code is to read the string to check wheather it is crossing the limit or not. If crossing the limit, then an error and else write in eeprom. That will be a simple comparison and if then else. Anyone can do that.

My plan is to like this:

#define SERVER_ADDR_SIZE 34
#define SERVER_ADDR_OFFSET 35
#define NEXT_STRING_SIZE (SERVER_ADDR_OFFSET+100)
#define NEXT_STRING_OFFSET (NEXT_STRING_SIZE+1)

here we consider that SERVER_ADDR will be 100 bytes. Now,s why 100 bytes, it's just a basic guess, how long the address you want. This is normally a space wasting method, and using a dynamic memory allocation could be more space saving, but that needs shifting other memory contents to more forward in order to accommodate the current (shift str 2 to forward so str 1 will not overlap with str 2) and that's a tremendous job during each eeprom saving like the windows defragment. Why taking so much pain on bunch of strings variable length and when it needs to use dynamic memory alloc, on eeprom when you can use the SD card with FAT32... put your data into a txt file better when it needs a lot of strings. each string per line...

So till here, the topic I have decided to close with one condition, someone will improve this work further I hope...

I thank to all of you for participating in this thread and be the helping hand to develop the code.

Thanks once again.
Mishu~