next topic »

[Reply #15 on:]

Once the data gets into an SD card there just are not undetected write errors.

Noise on the SPI bus is be the only thing left. 

You can check that by enabling CRC on SPI transfers between the Arduino and SD. 

I added software CRC to SdFat so edit SdFatConfig.h at about line 35 and change USE_SD_CRC to 1 or 2.

Code:

/**
 * To enable SD card CRC checking set USE_SD_CRC nonzero.
 *
 * Set USE_SD_CRC to 1 to use a smaller slower CRC-CCITT function.
 *
 * Set USE_SD_CRC to 2 to used a larger faster table driven CRC-CCITT function.
 */
#define USE_SD_CRC 0

Then all transfers on the SPI bus will be CRC protected.  Calls to SdFat functions will fail with an error return.

[Reply #16 on:]

Thank you very much, I will try this out, because since I've disabled writing while gprs is on the device worked for more than 24 hours without any faults.

Will respond back with results soon!

[Reply #17 on:]

As a test, try running the GSM and the Arduino/SDcard on two separate batteries.

This will put to rest the problem area.

An barrier of opto isolators  between the two may help your problem.

Try to write a RESET message into the SDcard.

Your memory may be getting corrupted.

Good Luck

[Reply #18 on:]

Once the data gets into an SD card there just are not undetected write errors.

Noise on the SPI bus is be the only thing left. 

You can check that by enabling CRC on SPI transfers between the Arduino and SD. 

I added software CRC to SdFat so edit SdFatConfig.h at about line 35 and change USE_SD_CRC to 1 or 2.

Code:

/**
 * To enable SD card CRC checking set USE_SD_CRC nonzero.
 *
 * Set USE_SD_CRC to 1 to use a smaller slower CRC-CCITT function.
 *
 * Set USE_SD_CRC to 2 to used a larger faster table driven CRC-CCITT function.
 */
#define USE_SD_CRC 0

Then all transfers on the SPI bus will be CRC protected.  Calls to SdFat functions will fail with an error return.

I actually failed in attempting this, because this made my rom a bit larger than before and I guess that the flash memory was not enough for SdFat buffers, everything crashed in a reset loop...

Because of the fact that no bug occurs when I don't write on the SD while gprs is running leads me to thinking that the problem is not with the SdFat but with some sort of electromagnetic interference that messes up the SPI bus (that runs below the gprs board). I will try isolating some how the two parts of the device, because using AC/DC adapter with larger capacity didn't change anything, which removes the possibility of power issues causing voltage drops on the SPI.

As a test, try running the GSM and the Arduino/SDcard on two separate batteries.

This will put to rest the problem area.

An barrier of opto isolators  between the two may help your problem.

Try to write a RESET message into the SDcard.

Your memory may be getting corrupted.

Good Luck

The memory card works perfectly in other test applications.. It's probably ok. I will do some more tests and will feed back soon...

[Reply #19 on:]

I guess that the flash memory was not enough for SdFat buffers, everything crashed in a reset loop...

Flash use will not cause a crash in a reset loop.  If you are that close to running out of RAM, you will likely have a problem if you log data and run gprs at the same time.  A pin change interrupt in SoftwareSerial can cause a stack overflow while an SdFat function like remove() is executing.

If possible check the amount of free stack by adding this include:

Code:

#include <SdFatUtil.h>

And this print in setup()

Code:

  Serial.println(FreeRam());

You need 200-300 bytes of free RAM in addition to any you allocate in functions.

I looked at your SD module and these often fail. 

The problem is that these modules don't use proper level shifters on MOSI, SCK, and CS.  These signals should be converted from 5V to 3.3V with an IC based level shifter.  Most SD cards are not designed to accept 5V signals.

Too bad you can't use CRC on the SD to check for data transfer errors.  You can check for any detected SD problem like this:

Code:

if (sd.card()->errorCode()) {
  // print SD I/O error code
  Serial.println(sd.card()->errorCode(), HEX);
}

Here are typical SD modules with a level shifter in addition to a 3.3V regulator
http://www.gravitech.us/sdcaad.html
https://www.adafruit.com/products/254
http://www.pjrc.com/teensy/sd_adaptor.html

[Reply #20 on:]

Flash use will not cause a crash in a reset loop.  If you are that close to running out of RAM, you will likely have a problem if you log data and run gprs at the same time.  A pin change interrupt in SoftwareSerial can cause a stack overflow while an SdFat function like remove() is executing.

I am not using SoftwareSerial, I am using the internal Serial object to communicate with the gprs. The worst operation done in my application is one call of sprintf() over 69 chars array. I will check my ram usage with the function you provided, but I am sure there's not much left..

I looked at your SD module and these often fail. 

The problem is that these modules don't use proper level shifters on MOSI, SCK, and CS.  These signals should be converted from 5V to 3.3V with an IC based level shifter.  Most SD cards are not designed to accept 5V signals.

Thanks for the feedback, I will keep in mind replacing this.

By the way all I had time today to test was to enable writing while gprs is running again and test the application and the device with gprs module wired up away from the device, not stacked on top. The results are pretty optimistic for now, the device has been working for like 5 hours now with no fails! If it continues to work normally like that for 24 hours I will uncomment the deletion of files (to restore the original application) and test it again. If the second test is also positive, then the problem is 100% eletromagnetic interference.

Thanks for the attention last few days, you've really done an amazing job with this library and its very honorable that you support people with its usage!

Will feed back soon

[Reply #21 on:]

I am not using SoftwareSerial, I am using the internal Serial object to communicate with the gprs.

HardwareSerial also uses lots of stack for interrupts.  I assumed SoftwareSerial since many gprs libraries use it.

Are you checking all SdFat calls for error returns?

[Reply #22 on:]

The case could be considered closed.

When the GPRS shield is detached and wired from a distance the device works perfectly with the original version of my application, which caused the most problems. I have removed all debug delays I've placed after writing, I have re-enabled writing while GPRS is working and re-enabled file deletion after successful transfer. The device is working for 48 hours now, the condition of the SD card's fat16 file system is also normal (with the GPRS shield stacked on top the device would not last 10 minutes). Beware of using Efcom's SIM900 GPRS shield, it has bad power supply, unstable firmware and... as we can all see - makes a lot of electromagnetic interference... the manufacturer didn't bother to shield against noises, which could have destroyed other peripheral devices. Not only the SD uses the SPI bus, but also very expensive sensor equipment...

Thank you very much for the attention!