[Poorly Solved]Ethernet SD MEGA 2560 server, SD fails after ethernet/512bytes

Dear
I am quite new and trying to host a SD web server, using

  1. Arduino Mega 2560 R3
  2. Ethernet shield with micro SD R3
  3. Windows 8 + arduino 1.0.5+r2
  4. Standard ethernet library
  5. Standard SD library, or latest SDFat library, both give same result.

Using examples:
http://startingelectronics.com/tutorials/arduino/ethernet-shield-web-server-tutorial/SD-card-web-server/

  1. The ethernet server can be setup using arduino server.println method, without using SD
  2. The SD can be setup, and Serial.println htm contents to serial monitor without problem, even with files more than 512 bytes
  3. Combined ethernet and SD library, the SD reading fails after ethernet.begin call. Setting other device CS pins to high has no effect.
  4. SD files can be read if it is re-init again after ethernet.begin call.
  5. In the code, I used a 1024 byte text file consist of all “a” for debugging.
  6. In the setup, all 1024 bytes of “a” can be printed to monitor.
  7. In the loop, if client.write() are used, only 512 byte of “a” can be printed, after that endless byte of FF(255) will printed.
  8. Using freeMEM shows more the 6K free SRAM all the time.

At the end of the day, I can only serve a htm file on SD for file size less then 512 bytes.
I have search few days and has no result, please help. thanks!
The code

[code]#include <SPI.h>
#include <Ethernet.h>
#include <SdFat.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192, 168, 1, 177); // IP address, may need to change depending on network
EthernetServer server(80);  // create a server at port 80
SdFat sd;
SdFile webFile;

void setup()
{
    pinMode(53, OUTPUT); //for Mega 2560, but this line has no effect
    pinMode(4,OUTPUT);   //this line has no effect  
    pinMode(10,OUTPUT);  //this line has no effect
    digitalWrite(4,HIGH);//this line has no effect
    digitalWrite(10,HIGH);//this line has no effect
    
    Serial.begin(9600);       // for debugging
    
    // initialize SD card
    Serial.println("Initializing SD card...");
    if (!sd.begin(4,SPI_HALF_SPEED)) {
        Serial.println("ERROR - SD card initialization failed!");
        return;    // init failed
    }
    Serial.println("SUCCESS - SD card initialized.");
    // check for a.txt file
    if (!webFile.open("a.txt",O_READ)) {
        Serial.println("ERROR - Can't find a.txt file!");
        return;  // can't find index file
    }
    {
      Serial.println("SUCCESS - Found a.txt file.");
      webFile.close();
    }
    
    digitalWrite(4,HIGH); //this line has no effect
    Ethernet.begin(mac, ip);  // initialize Ethernet device
    server.begin();           // start to listen for clients
    Serial.print("server is at ");
    Serial.println(Ethernet.localIP());
    
    digitalWrite(10,HIGH); //this line has no effect
    //debug code to dump the text file in setup()
    sd.begin(4,SPI_HALF_SPEED);//need to re-init SD after ethernet functions, otherwise SD won't work after Ethernet.begin call
    webFile.open("a.txt",O_READ); Serial.println("file opened");
    byte data;
    while (webFile.available()) {
      data = webFile.read();
      Serial.write(data);
    }
    webFile.close();Serial.println("\n\rfile closed");

}

void loop()
{
    EthernetClient client = server.available();  // try to get client

    if (client) {  // got client?
        boolean currentLineIsBlank = true;
        while (client.connected()) {
            if (client.available()) {   // client data available to read
                char c = client.read(); // read 1 byte (character) from client
                Serial.write(c);
                // last line of client request is blank and ends with \n
                // respond to client only after last line received
                if (c == '\n' && currentLineIsBlank) {
                    // send a standard http response header
                    client.println("HTTP/1.1 200 OK");
                    client.println("Content-Type: text/html");
                    client.println("Connection: close");
                    client.println();
                    Serial.println("Sending web page");
                    
                    sd.begin(4,SPI_HALF_SPEED);//need to re-init SD after ethernet functions, otherwise SD won't work 
                    webFile.open("a.txt",O_READ); Serial.println("file opened");
                    byte data;
                    while (webFile.available()) {
                      data=webFile.read();
                      Serial.write(data);
                      client.write(data);  // only 514 bytes can be read if using this line
                    
                    }
                    webFile.close();Serial.println("file closed");
                    break;
                }
                // every line of text received from the client ends with \r\n
                if (c == '\n') {
                    // last character on line of received text
                    // starting new line with next character read
                    currentLineIsBlank = true;
                } 
                else if (c != '\r') {
                    // a text character was received from client
                    currentLineIsBlank = false;
                }
            } // end if (client.available())
        } // end while (client.connected())
        delay(1);      // give the web browser time to receive the data
        client.stop(); // close the connection
    } // end if (client)
}

[/code]

The output

Initializing SD card...
SUCCESS - SD card initialized.
SUCCESS - Found a.txt file.
server is at 192.168.1.177
file opened
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

file closed
GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: zh-TW
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0; MAMIJS)
Accept-Encoding: gzip, deflate
Host: 192.168.1.177
DNT: 1
Connection: Keep-Alive

Sending web page
file opened
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ

a.txt (1 KB)

It is too confusing.

The SD card is connected to pin 4 for Chip-Select 4 ? And the Ethernet to pin 10 ? In setup() you set 4,10,53 as OUTPUT, and 4,10 HIGH. That is okay.

But later in the sketch, please don't do that anymore. The SD library and the Ethernet library will take care of those signals. Open and close the file only when it is going to be read. You can check if the file exists with: http://arduino.cc/en/Reference/SDexists

You might even remove the HALF_SPEED parameter, and use the default SD library (not the sdFat).

I have no loop to check available() while transferring a file. I use this: int length = webFile.available(); And then I just copy the webFile in a for-loop for the length. There should not be a 512 byte boundery.

I know you tried to make it work by re-initializing the SD and the chip-select signals. But you have to remove that and try to find the problem. You have now planted too many trees so you can't see how to get out of the woods.

It is an official Ethernet Shield, that is compatible with 5V Arduino ? You could try Arduino 1.5.7 BETA. You can download it, unzip it in a folder, start it, and set the projects folder. After that you can still use the older Arduino IDE and also the newer one.

I use a Mega with the ethernet shield and I do not have the problem you describe. Here is my code. http://playground.arduino.cc/Code/WebServerST

This code does not send one byte at a time. It sends the data in 64 byte packets. Look for this code:

                   while(myFile.available()) {
                    tBuf[clientCount] = myFile.read();
                    clientCount++;
                    tBuf[clientCount] = 0;

                    if(clientCount > 63) {
                      client.write((byte*)tBuf,64);
                      clientCount = 0;
                    }

                  }
                  if(clientCount > 0) {
                    client.write((byte*)tBuf,clientCount);
                  }
                }

                myFile.close();

I send pictures with this and it does fine.

SurferTim, that piece of code is not the problem, and the buffer optimization makes it harder to understand :| I think that georgychen does too many initializations of the SD card, and perhaps doing something with the chip-selects has somehow influence.

@Peter_n: My code works. That was the point. It doesn't do anything with the SPI slave selects that doesn't need to be done, and it initializes the SD card only once.

If complexity scares the OP, then maybe this is not not right project to undertake.

Yes, I get your point. Let georgychen first make a nice clean sketch. There might even be a hardware problem.

Thanks for your quick reply.
Peter_n

The SD card is connected to pin 4 for Chip-Select 4 ? And the Ethernet to pin 10 ?
It is an official Ethernet Shield, that is compatible with 5V Arduino ?

It is a arduino ethernet shield r3 with built in micro SD. It has the ICSP header. Pin 10 is for Ethernet CS, pin 4 is for SD CS. Both operate good separately. However, it is a compatible shield, not official. My mega 2560 is also a compatible. I am trying to narrow down to see if it is a hardware problem. When I go back home, I’ll try on Leonardo (also a compatible :sweat_smile:) and IDE beta.

You might even remove the HALF_SPEED parameter, and use the default SD library (not the sdFat).

I did used the default SD library initially, the author of the sdFat (which SD lib was based on) said the new sdFat lib prevent some SPI problems, by setting proper parameters not only on the begin call, but also on other methods. However, my problem exists using both SD and sdFAT lib.

Yes, it is root problem to initialize SD more than once. I suspect the client.write breaks the SD read in the first loop, but the file hold 512 bytes because of SD reads a block of 512 bytes each time, and therefore only 512 byte of file can be read/sent.

SurferTim
I tried your code, and here is the output result:

Starting SD..ok
Ready

Client request #1: GET / HTTP/1.1
file = /
file type = 
method = GET
params = 
protocol = HTTP/1.1
Home page SD file
filename format ok
File not found
disconnected

Client request #2: GET /index.htm HTTP/1.1
file = /INDEX.HTM
file type = HTM
method = GET
params = 
protocol = HTTP/1.1
SD file
filename format ok
File not found
disconnected

Client request #3: GET /a.txt HTTP/1.1
file = /A.TXT
file type = TXT
method = GET
params = 
protocol = HTTP/1.1
SD file
filename format ok
File not found
disconnected

Browser requested three times, 1st without filename, second using http://x.x.x.x/index.htm, 3rd using a.txt. I am pretty sure these file are in the SD card, and I can find/dump these files using another SD only (no ethernet) sketch.

To narrow down, I used your new web server code and modified to a bare-bone:

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
#include <utility/w5100.h>
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xEC };
IPAddress ip( 192,168,1,177);
EthernetServer server(80);

void setup()
{
  pinMode(53,OUTPUT);
  Serial.begin(9600);

  // disable w5100 SPI while starting SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);
  Serial.print(F("Starting SD.."));
  if(!SD.begin(4)) 
    Serial.println(F("failed"));
  else 
    Serial.println(F("ok"));
  dumpfile("a.txt");
  
  Ethernet.begin(mac, ip);
  digitalWrite(10,HIGH);
  delay(2000);
  server.begin();
  Serial.println(F("Ready"));
  dumpfile("a.txt");
}
void loop()
{
}

void dumpfile(char* filename)
{
  Serial.println(filename);
  
  File webFile;
  if (SD.exists(filename)){
    Serial.println(F("File found."));
    webFile = SD.open(filename);
    if (webFile) {
      Serial.println(F("Sending SD file..."));
      while(webFile.available()) {
        Serial.write (webFile.read()); //send SD file content to serial monitor
      }
    Serial.println("");
    webFile.close();
    }//end if(webFile)
  }//end if (SD.exists(filename))
  else
    Serial.println(F("File not found."));
}

This sketch does four things: Initiate SD card, dump the text file in SD, initiate ethernet, dump the same file again.
Here’s the result:

Starting SD..ok
a.txt
File found.
Sending SD file...
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Ready
a.txt
File not found.

The second try on SD.exists fails. Apparently this is not related to the byte by byte reading. I agree that the multiple SD initialize walk-around is a bad idea, and this walk-around can’t meet my project need. Lower-level programming is also something I know very less.

I guess the SD SPI is broken by ethernet calls, since re-initialize SD will temporary re-establish SD communication. Any ideas to verify that?

It the hardware is okay, using Ethernet should never break the SD interface. When you change the sketch to avoid that, your sketch will become weird and you can never return to a situation that is working well. If your sketch is okay, try different hardware.

You still set pin 10 high [u]after[/u] Ethernet.begin(). Please don't do that. The Ethernet library takes over the control of the chip-select.

I prefer this: pin 4,10,53 as output, and set all HIGH to disable all chip-selects. After that SD.begin and Ethernet.begin.

The Mega 2560 R3 and Uno R3 are compatible with the Ethernet shield R3. The Mega 2560 R2 and Uno R2 are compatible with the Ethernet shield R2.

Can you have a good look at that clone/fake Ethernet module ? It is R3 compatible ? Perhaps there is a shortcut on it, or you don't have enough 3.3V power. Can you make a photo of it ?

You still set pin 10 high after Ethernet.begin(). Please don't do that. The Ethernet library takes over the control of the chip-select.

That is my fault. It is a compensation for a bug in an old version of the ethernet library. I will correct that in the playground version of my sketch.

edit: Here is that code's output if the files exist on the SD card.The index.htm file loads a defcss.css file, and the web browser loads the favicon.ico file.

Starting SD..ok 192.168.2.2 Ready

Client request #1: GET / HTTP/1.1 file = / file type = method = GET params = protocol = HTTP/1.1 Home page SD file filename format ok SRAM = 6114 file found..opened..send..closed disconnected

Client request #2: GET /defcss.css HTTP/1.1 file = /DEFCSS.CSS file type = CSS method = GET params = protocol = HTTP/1.1 SD file filename format ok SRAM = 6114 file found..opened..send..closed disconnected

Client request #3: GET /favicon.ico HTTP/1.1 file = /FAVICON.ICO file type = ICO method = GET params = protocol = HTTP/1.1 SD file filename format ok SRAM = 6114 file found..opened..send..closed disconnected

I prefer this: pin 4,10,53 as output, and set all HIGH to disable all chip-selects. After that SD.begin and Ethernet.begin.

The modified sketch is as follow;

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
#include <utility/w5100.h>
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xEC };
IPAddress ip( 192,168,1,177);
EthernetServer server(80);

void setup()
{
  pinMode(4,OUTPUT);
  pinMode(10,OUTPUT);
  pinMode(53,OUTPUT);
  digitalWrite(4,HIGH);
  digitalWrite(10,HIGH);
  digitalWrite(53,HIGH);
  
  Serial.begin(9600);
  
  Serial.print(F("Starting SD.."));
  if(!SD.begin(4)) 
    Serial.println(F("failed"));
  else 
    Serial.println(F("ok"));
  
  Serial.println(F("1st dump after SD.begin"));
  dumpfile("a.txt"); 
    
  Ethernet.begin(mac, ip);
  delay(2000);
  server.begin();
  Serial.println(F("Ready"));
  
  Serial.println(F("2nd dump after initialize ethernet.begin"));
  dumpfile("a.txt"); 
  
  //initilize SD the second time, which shouldn't be required
  Serial.print(F("Starting SD.."));
  if(!SD.begin(4)) 
    Serial.println(F("failed"));
  else 
    Serial.println(F("ok"));
  
  Serial.println(F("3nd dump after re-Sd.begin"));  
  dumpfile("a.txt");
}
void loop()
{
}

void dumpfile(char* filename)
{
  Serial.println(filename);
  
  File webFile;
  if (SD.exists(filename)){
    Serial.println(F("File found."));
    webFile = SD.open(filename);
    if (webFile) {
      Serial.println(F("Sending SD file..."));
      while(webFile.available()) {
        Serial.write (webFile.read()); //send SD file content to serial monitor
      }
    Serial.println("");
    webFile.close();
    }//end if(webFile)
  }//end if (SD.exists(filename))
  else
    Serial.println(F("File not found."));
  
  Serial.println("");
}

Three pins (4,10,53) were set output, high in the first place. First dump on SD file is ok. After ethernet.begin, second dump on SD fails. The weird thing is, although re-sd.begin returns fail, SD file can be read again on the 3rd dump.

Output result:

Starting SD..ok
1st dump after SD.begin
a.txt
File found.
Sending SD file...
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Ready
2nd dump after initialize ethernet.begin
a.txt
File not found.

Starting SD..failed
3nd dump after re-Sd.begin
a.txt
File found.
Sending SD file...
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

And I comment out the whole pin stuff, the result is still the same as the output above

/*pinMode(4,OUTPUT);
  pinMode(10,OUTPUT);
  pinMode(53,OUTPUT);
  digitalWrite(4,HIGH);
  digitalWrite(10,HIGH);
  digitalWrite(53,HIGH);*/

My situation is somewhat similar to this topic,

but cannot find conclusive reason yet.

Make the sketch cleaner by removing the second init, and try to find how the hardware is connected. Or you could make a sketch that lowers pin 4 for 5 seconds and after that pin 10 for 5 seconds with led on. Try to see if they pass through the logic gate on the ethernet board which converts them to 3.3V.

To be able to help you, we need links to your ethernet board, and very sharp photos of both sides, and photos how you have connected it. You might be using the wrong ethernet shield (known problem), or there is too little solder used (I have had that on cheap board from Ebay), or there is a shortcut on the board (I had solder shortcutting two pins), or a power issue (can happen), or ... something else.

Peter_n

The Mega 2560 R3 and Uno R3 are compatible with the Ethernet shield R3.
The Mega 2560 R2 and Uno R2 are compatible with the Ethernet shield R2.

Can you have a good look at that clone/fake Ethernet module ? It is R3 compatible ? Perhaps there is a shortcut on it, or you don’t have enough 3.3V power. Can you make a photo of it ?

The boards both claims R3, and the pin layout looks like R3, they stacks perfectly. I didn’t use any jumper lines for this sketch, as shown in the picture. For the power, currently both board are powered by computer usb. At home, I have tried an regulated 5v input with 1.5 A capacity. Result is the same.

I doubt the power will be an issue, because they works fine separately, but thanks for the advise. Currently I am out in hotel so I don’t have a multimeter. Will do it back home.

SurferTim
Thanks for your code, I will definitely use it if this problem can be fixed. There are many tutorials about adruino SD web server, and I am definitely sure that the problem is lying somewhere on my desk. I want to fix this instead of bypass this. Still need your advises…

I am still scratching my head thinking of any possibility of the ethernet library interacts(breaks) with SD library. :frowning: :frowning:

To be able to help you, we need links to your ethernet board, and very sharp photos of both sides, and photos how you have connected it.

Here are the links, sorry that they are in Chinese
ethernet r3: http://goods.ruten.com.tw/item/show?21305118122242
Mega 2560 r3: http://goods.ruten.com.tw/item/show?21303293227330

And the photos attached:

Thanks for the photos. I can't see anything wrong with them. And I'm out of ideas.

You can try the 1.5.7 BETA version. But I doubt if that makes a difference.

My best advise is to buy a new Mega and a new Shield. Either the official ones (to keep away from troubles for sure), or clones from a different manufacturer. A Mega 2560 R3 clone is less than 13 dollars. You can buy the CH340/CH341 version if you are willing to install a driver from wch-ic.com An Ethernet Shield W5100 R3 clone is less then 8 dollars.

It is possible that a batch of shields was rejected in the factory, and that they end up on Ebay somehow.

You're right, I just installed 1.5.7 beta, and the result is the same. SD library functions stops working after ethernet calls, such as client.write(). I'll try it on Leonardo tomorrow. Thanks!

Finally got this worked in a dirty way:

  1. SD was initialized after ethernet.
  2. Each read is limited to a block of 512 bytes
  3. SD was initialized for each read, file is opened and closed for each read.

Not a good way, but worked somehow. And reading/sending in 512 bytes block for a 10k web page reduces the loading time from 3xxx millisecond to 8xx milliseconds, about 3-4 times faster than byte to byte sending. Thanks to Peter_n and SurferTim.

Here’s the code

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xEC };
IPAddress ip( 192,168,1,177);
EthernetServer server(80);

void setup()
{
  Serial.begin(9600);
  Ethernet.begin(mac, ip);
  delay(2000);
  server.begin();
  
  Serial.println(F("Server ready"));
  Serial.print(F("Starting SD.."));
  if(!SD.begin(4)) 
    Serial.println(F("failed"));
  else 
    Serial.println(F("ok"));

  
}

void loop()
{
    EthernetClient client = server.available();  // try to get client
    if (client) {  // got client?
        boolean currentLineIsBlank = true;
        File webFile;
        long t;
        char readbyte;
        char tBuf[512];
        unsigned int byteCount=0;
        unsigned int filesize;
        unsigned int loopCount=0;
        unsigned int loopTotal=0;
        char sdfilename[]= "gauge.htm";
        while (client.connected()) {
            if (client.available()) {   // client data available to read
                char c = client.read(); // read 1 byte (character) from client
                // last line of client request is blank and ends with \n
                // respond to client only after last line received
                if (c == '\n' && currentLineIsBlank) {
                    // send a standard http response header
                    client.println("HTTP/1.1 200 OK");
                    client.println("Content-Type: text/html");
                    client.println("Connection: close");
                    client.println();
                    // send web page
                    t=millis();                    
                    SD.begin(4); 
                    webFile = SD.open(sdfilename);
                    if (webFile){
                    filesize = webFile.available();
                    Serial.print(F("Sending SD file..."));
                    Serial.print(filesize);
                    Serial.println(F(" bytes"));
                    webFile.close();
                    loopTotal = filesize>>9;
                    //read and send data in blocks of 512 bytes
                    for (loopCount=0;loopCount<loopTotal;loopCount++){
                    SD.begin(4);
                    webFile = SD.open(sdfilename);
                    webFile.seek(loopCount<<9);
                      for (int x=0; x<512;x++ ){
                        tBuf[x]=webFile.read();
                      }
                    client.write ((byte*)tBuf,512);
                    filesize=filesize-512;
                    webFile.close();
                    }// end for (loopCount)
                    
                    if (filesize>0){
                      SD.begin(4);
                      webFile = SD.open(sdfilename);
                      webFile.seek(loopTotal<<9);
                      for (int x=0; x<filesize;x++ ){
                       tBuf[x]=webFile.read();
                      }
                      client.write ((byte*)tBuf,filesize);
                      webFile.close(); 
                    }
                    t=millis()-t;
                    Serial.print(F("Page transfer time "));
                    Serial.print(t);
                    Serial.println(" milliseconds.");
                    Serial.println(F(""));
                    }//end if loop
                    break;
                }
                // every line of text received from the client ends with \r\n
                if (c == '\n') {
                    // last character on line of received text
                    // starting new line with next character read
                    currentLineIsBlank = true;
                } 
                else if (c != '\r') {
                    // a text character was received from client
                    currentLineIsBlank = false;
                }
            } // end if (client.available())
        } // end while (client.connected())
        delay(1);      // give the web browser time to receive the data
        client.stop(); // close the connection
    } // end if (client)
}

output:

Server ready
Starting SD..ok
Sending SD file...10700 bytes
Page transfer time 893 milliseconds.

Sending SD file...10700 bytes
Page transfer time 892 milliseconds.

And below is the web page screen shot. The dial meter credits to:

http://startingelectronics.com/tutorials/arduino/ethernet-shield-web-server-tutorial/SD-card-gauge/

gauge.htm (10.4 KB)

I’m glad you have it (poorly) working, but I hope you buy another Ethernet Shield, so you can compare them.

Same goes to me… i managed to solve my problem using georgychen’s dirty way. :slight_smile:
The problem will start to happen even when i just call the EthernetServer server(80). The sd will failed to initialize.