Strange problem with SD card on Arduino Mega and Ethernet shield

Hi all
Sorry for my first post being a question.

I am doing quite well writing sketches for the Arduino and have built a number of projects. I am really suffering with this one though.

I have an Arduino Mega 2560 with a Mega compatable Ethernet shield connected.

I have a microSD card that has four files on it :-
index.htm
segment.js
favicon.ico
water.jpg

These are used to serve a webpage from the Arduino when a client first connects. The page is then updated every second with an XML burst from the Arduino.

Now all this works fine except the SD card drives me mad.

If I startup the unit from cold the SD card is recognised but I cant see the files
If I reset the Arduino or the Ethernet card with their buttons, the SD is recognised but I cant see the files.
If I remove and re-insert the SDcard with the Arduino and Ethernet shield powered up, then press the reset button I can see all the files on the card and everything is lovely.
BUT, if I now reset either the Arduino or the Ethernet shield, I can no longer access the files till I remove and re-insert the card, then reset again.

This is driving me nuts.
I have tried using the standard Arduino SD library, with SDbegin(4) to start the SD library and SD.exists(filename) to check if the files are there
I then tried using SdFat (which I find harder to understand) and get the same problem

So I have now created a tiny sketch that just takes the SD setup and file checks routines which still suffers exactly the same way. Code attached below.

#include <SPI.h>
#include <Ethernet.h>

#include <SdFat.h>
SdFat sd;
SdFat file;



void setup() {
  // disable Ethernet chip
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH),

  Serial.begin(9600);       // for debugging
  //check for the SD card and all the files. If they are not present then exit.
  
 
  //try to initialise the card
  if (!sd.begin(4)) {
    Serial.println("ERROR - SD card initialization failed!");
  }
  else Serial.println("SUCCESS - SD card initialized.");
  
  //check for index.htm file
  if (!sd.exists("index.htm")) {
    Serial.println("ERROR - Can't find index.htm file!");
  }
  else Serial.println("index.htm found");
  
  if (!sd.exists("favicon.ico")) {
    Serial.println("ERROR - Can't find favicon.ico file!");
  }
  else Serial.println("favicon.ico found");
  
  if (!sd.exists("segment.js")) {
    Serial.println("ERROR - Can't find segment.js file!");
  }
  else Serial.println("segment.js found");
  
  if (!sd.exists("water.jpg")) {
    Serial.println("ERROR - Can't find background.png file!");
  
  }
  else Serial.println("favicon.ico found");
  
}

void loop(){
}

When I reset the cards with the SD inserted I get this output on the serial monitor
SUCCESS - SD card initialized.
ERROR - Can't find index.htm file!
ERROR - Can't find favicon.ico file!
ERROR - Can't find segment.js file!
ERROR - Can't find background.png file!

no matter how many times I reset I get the same.

BUT, remove and re-insert the SD card and then reset the Arduino and I get this...
SUCCESS - SD card initialized.
index.htm found
favicon.ico found
segment.js found
favicon.ico found

but the very next reset without removing and inserting the card goes back to the errors again.

Please could anybody give me some advice on how I can get the card and all it's files recognised on every reset?

Many many thanks in advance
Julian

Your program works on an Arduino Mega with a Arduino Ethernet shield and a SanDisk microSD.

I edited this part of your program to use "background.png" for all strings.

  if (!sd.exists("water.jpg")) {
    Serial.println("ERROR - Can't find background.png file!");
 
  }
  else Serial.println("favicon.ico found");

Here is the output after several resets on the Mega and Ethernet shield.

SUCCESS - SD card initialized.
index.htm found
favicon.ico found
segment.js found
background.png found
SUCCESS - SD card initialized.
index.htm found
favicon.ico found
segment.js found
background.png found
SUCCESS - SD card initialized.
index.htm found
favicon.ico found
segment.js found
background.png found
SUCCESS - SD card initialized.
index.htm found
favicon.ico found
segment.js found
background.png found

Try another SD card. I have seen SD cards that will not do a software reset and need to have power cycled.

If possible try the program with another SD shield/module. Seems very unlikely the problem is with the Ethernet shield but who knows.

I am using the latest SdFat beta since I wrote SdFat.

Hi,
Many thanks for that. I have a Samsung microSD card inserted. Not sure if I have another brand microSD around but can have a look.
To try another Ethernet/SD shield means buying one since I only have one.

I take it that I have all the setup OK in my sketch (pin numbers etc) as yours seems to work OK.

I have tried uploading the Sdinfo sketch from the SdFat examples exactly as it is.
Running that, with the card inserted gives

SdFat version: 20150324

Assuming the SD is the only SPI device.
Edit DISABLE_CHIP_SELECT to disable another device.

Assuming the SD chip select pin is: 53
Edit SD_CHIP_SELECT to change the SD chip select pin.

type any character to start

init time: 3 ms

Card type: SDHC

Manufacturer ID: 0X0
OEM ID:
Product:
Version: 0.0
Serial number: 0X777D0000
Manufacturing date: 11/2013

cardsize: 33554.43MB (MB=1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBloc: true
OCR: 0XC0FF8000

No MBR. Assuming Super Floppy format.

Volume is FAT32
blocksPerCluster: 32
clusterCount: 2046976
freeClusters: 2046968
freeSpace: 33537.52 MB (MB = 1,000,000)
fatStartBlock:782
fatCount : 2
blocksPerFat: 15993
roorDirStart: 2
dataStartBlock: 32768

type any character to start

This seems to run happily no matter how many times I reset the Arduino with no need to remove and re-insert the card

You may have a counterfeit SD.

Here is what a Samsung 32GB card looks like:

init time: 2 ms

Card type: SDHC

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0
Serial number: 0XFC5E3635
Manufacturing date: 4/2014

cardSize: 32009.36 MB (MB = 1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBlock: true
OCR: 0XC0FF8000

SD Partition Table
part,boot,type,start,length
1,0X0,0XC,8192,62510080
2,0X0,0X0,0,0
3,0X0,0X0,0,0
4,0X0,0X0,0,0

Volume is FAT32
blocksPerCluster: 64
clusterCount: 976464
freeClusters: 976310
freeSpace: 31991.73 MB (MB = 1,000,000 bytes)
fatStartBlock: 9318
fatCount: 2
blocksPerFat: 7629
rootDirStart: 2
dataStartBlock: 24576

I have never seen a Samsung card with this data missing:

Manufacturer ID: 0X0
OEM ID:
Product:
Version: 0.0

Here are five Samsung cards:

High end 32GB Samsung:

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0

High end 16GB Samsung:

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0

Low end 8GB Samsung:

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0

Middle grade 16GB Samsung:

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0

Middle grade 32 GB Samsung:

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0

I don't know how your card was formatted but is is non-standard. All cards with standard formatting have an MBR:

No MBR. Assuming Super Floppy format.

32GB cards should have 64 blocks per cluster:

blocksPerCluster: 32

You might try the standard Formatter.

Probably won't help.

Hi,
Many thanks for the help, I really appreciate it
The card was formatted with Windows 7. I have now used the formatter you linked to, copied the files back onto the card and run the sketch again.
Healthier results this time but still no ID

type any character to start

init time: 2 ms

Card type: SDHC

Manufacturer ID: 0X0
OEM ID:
Product:
Version: 0.0
Serial number: 0X777D0000
Manufacturing date: 11/2013

cardsize: 33554.43MB (MB=1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBlock: true
OCR: 0XC0FF8000

SD Partition Table
part,boot,type,start,length
1,0X0,0XC,8192,65527808
2,0X0,0X0,0,0
3,0X0,0X0,0,0
4,0X0,0X0,0,0

Volume is FAT32
blocksPerCluster: 64
clusterCount: 1023616
freeClusters: 1023610
freeSpace: 33541.65 MB (MB = 1,000,000)
fatStartBlock:8580
fatCount : 2
blocksPerFat: 7998
rootDirStart: 2
dataStartBlock: 24576

type any character to start

Healthier results this time but still no ID

You can't fix those fields. You just have a fake card.

It is amazing how many fake cards are floating around. I got a fake SanDisk card from an Amazon seller. I discovered it in performance tests. It had binary zeros in the same four fields that are bad in your card.

It is common for fake cards to have binary zeros in those four fields.

That is annoying as I have had it for some time now and paid decent money for it online.
I will try another card when I can lay my hands on one.

I have discovered that it is only the reset which plays up. If I power cycle the whole Arduino then the server starts up OK.

Here is the webpage that it is serving at the moment Arduino Pool Controller
If the Arduino is up you should see the page

The background, favicon and javascript are all loaded at page load time from the SD card on the Arduino to your browser.
Then every second the browser requests the values (pool temp, return temp, pump status, boiler status and mode) from the Arduino which is supplies as XML.
The two javascript LED displays and the text are updated.

This clearly is running on the real pool cotroller, it is running on a spare (development) Arduino whilst I get the code right so the values just cycle from 0 to 100C
I have another Arduino currently running the real pool controller and doing a good job, about 2000 line sketch which allows three independant time periods per day with different temperature setpoints, a manual mode, hysteresis, radio remote control and an LCD display.
The ethernet addition is the final part to monitor the controller remotely (ie over my wifi or the internet) and change settings if needed.
It is getting really hard now though as whilst I am happy with writing code in C, the wonders of HTML, CSS and Javascript are really taxing my limited brain power and the SD card issue on top was just getting too much.