Could Bill Greiman take a look at this ?

I started this SD "problem" in the Programming section, so hope that this doesn't amount to cross-posting (maybe a moderator could move the thread into this section ?)

Original thread at http://forum.arduino.cc/index.php?topic=285771.15

I have no idea what the program should do. I put the following at the start of the program so it does not run before the Serial monitor is open.

  Serial.println("type any char to start");
  while (Serial.read() < 0) {}

When it runs it prints this:

type any char to start
============ Creating G.tmp ==================
Writing to G.tmp
chFileMode = C
File does not exist - G.tmp
Record is - Some text

------- Start of display of G.tmp - 1 -------
00:00 - G1Some text
------- End of display of G.tmp --------

******* error opening E.tmp ********

============ Creating E.tmp =============
Writing to E.tmp
chFileMode = C
File does not exist - E.tmp
Record is - And some other text

------- Start of display of G.tmp - 3 -------
00:00 - G1Some text
------- End of display of G.tmp --------

------- Start of display of E.tmp - 4 -------
00:00 - E2And some other text
------- End of display of E.tmp ------
Finished creating temporary logs

This is E.tmp

00:00 - E2And some other text

This is G.tmp

00:00 - G1Some text

What should the program do?

Hello Bill,
As you will have seen from the late entries in the thread in the Programming section, I have found that the problem is being introduced by the microSD card itself ! It therefore doesn’t surprise me that, in running the code, the results are good.
For yor interest, I will paraphrase the original problem:-

I am currently working on a heating control system and the code currently stands at over 70KB. Within this design there is a logging system, but before establishing the time and date, I have to produce temporary logs to catch early events. (G.tmp and E.tmp for 'G’eneral messages and E.tmp for 'Error messages). I recently noticed what appeared to be some corruption of the temporary logs, and luckily also in the actual creation of the files. I therefore extracted the common function used to create and add messages to the particular logs, and built a test sketch around it. I shall describe the LAST version of this

The failing scenario was that the message added to E.tmp also appeared to get added to G.tmp within the SAME access to the SD card. (This was proved by adding a two character prefix to the message being stored within the function itself - G1 implies a call for the General file and the first usage of the function, E2 is call for the Error file and the second use of the function)
The sequence used in the final version of the test sketch was
Write General file
Read General file
Write Error file
Read Error file
Re-read General file

The prefix charaters on the Re-read showed that the General file had been corrupted by the previous write to the Error file.

I hope that this description helps !

Since replacing the microSD card with another, the test sketch runs successfully !

Hi again Bill,
Sorry, I should have attached the final test sketch and the associated Serial Monitor output for your convenience. Incidently, since running this, I upgraded to SdFat but this made no difference ! I can’t remember how the “failing” card was formatted, but I guess this will be my next step. The card has been working perfectly for some time, so this problem came as a bit of a surprise !!

NewSketch.txt (7.54 KB)

NewSketchSM.txt (737 Bytes)

You will get corrupt cards from any library if you allow the program to start then initialize the serial monitor which reboots the Arduino.

What happens is that a file is partially created or in the process of a write when the reboot happens. It is then possible for the same cluster to be allocated to two files. This will cause the problem you have.

You must make sure all files are properly close before rebooting.

Your card is probably not bad. Your program is bad.

Hello Bill,
Actually, my full heating control system sketch DOES take precautions regarding ensuring the closure of any open files (e.g. due to power failure, or powering down for any other reason)

I’m not sure that I follow your argument about delaying execution of the sketch by “while( Serial.read() <0){} at the start of setup(). How does this delay safeguard any file operation (i.e. ensure closure) which was “interrupted” by the previous reboot ? Surely the damage has already been done ?

I’m not sure that I follow your argument about delaying execution of the sketch by "while( Serial.read() <0){} at the start of setup(). How does this delay safeguard any file operation (i.e. ensure closure) which was “interrupted” by the previous reboot ? Surely the damage has already been done ?

As soon as power is applied to or after you reload a program in an Arduino it starts executing. When you bring up the serial monitor, the Arduino is reset by a signal over the USB. This is an emulated serial CTS, clear to send, pulse. So at some random place in the program a reset occurs.

If the program was doing a file operation, this may cause corruption of the SD. That happened the first time I ran the example you posted.

Inserting this

while( Serial.read() <0){}

Prevents the program from executing before the serial monitor is open. Serial.read() returns -1 when no data is available so the program stalls at that point and no file operations happen until you enter a character.

Recent Arduino Unos an Megas have a solder bridge you can cut to prevent this auto reset. The solder bridge is labeled with “RESET EN”.

If you notice, most of my SdFat example that create and write files have the above wait for serial input because of this problem.

Your program seems to be more vulnerable than most, probably since it deletes and creates files at just the right time to be caught by the unexpected reset.

Here is a typical problem that this reset causes http://electronics.stackexchange.com/questions/24743/arduino-resetting-while-reconnecting-the-serial-terminal

Hi Bill,
My problem is that my heating control sketch will normally run without the USB connected, but I still want the Serial interface available should circumstances demand it’s use (for debug). Any suggestions ? (Obviously cutting that bridge or use of the 22uF cap are possibilities to avoid using the “hang” solution)

Incidently, many thanks for your input on all of this ! Much appreciated !

Update:-
For completeness, I have just added the SM wait to the test sketch and inserted my “bad” microSD card (sketch and SM output files attached). It STILL fails, so I don’t reckon this is due to the SM reset !

20141217_Sketch.txt (7.93 KB)

20141217_SM.txt (762 Bytes)

I assume you reformatted the card with the SD formatter from the SD association.

Bad SD cards are very rare. They have great ECC and remap bad spots.

Don't trust OS utilities to format SD cards. They often don't do a from scratch format, just clear existing areas rather than using the correct layout defined by the SD standard.

Some cards appear to be bad on Arduino but this is usually a SPI problem. No CRC is used for the data transfer so bad wiring or interference from another SPI device can cause problems. Some cards may be more susceptible than others. SdFat has software CRC as an option and this can detect this problem.

Hi Bill,
Ooops !! Forgot to do that in my haste !
Used SD Formatter to do a full erase.
Reran my test sketch 3 times - all perfectly correct !

Coming back to my last question, do you reckon that the 22uF capacitor would be the best solution for my full heating control system sketch ?

Once again, thanks for your magnificent help with this issue !

I like the idea of the 22 μF cap. It is easy to remove or you can provide a jumper or switch to disable it. Just test it by connecting the Serial monitor to make sure it is reliable.

I have cut the reset enable solder bridge for one application and it works. The down side is that it is then a bit tricky to load a new program since the auto reset by the loader no longer works. I just wait until the IDE says loading and hit the reset button but this doesn't always work.

I have done custom AVR boards with a jumper for auto reset and I like that best.

Hi Bill,
Missed your reply, and in the meantime, implemented the 22uF fix and it works fine !!! Means another switch on the interconnect card (from the Mega to the relay card) in my heating project, but what the hell !! In fact, the bonus is that I can now connect a PC to the serial port and see what is currently going on, whereas before, it would cause a reset and destroy the current 'environment'

Hey, thanks SO MUCH for your help - it REALLY was appreciated !!

Best wishes,
Doug