Problem using both SPI library and SD library in Arduino UNO program

[Dave2Satellite]

Hello everyone,

Our goal is to use the Arduino UNO to get data from a sensor using SPI and then write that data to a file on a micro-SD card (which also uses SPI).  I am having problems using SPI to communicate with two SPI devices: the adafruit micro-SD card breakout board+ and a YEI 3-Space Sensor Embedded (Inertial Measurement Unit / Attitude & Heading Reference System).

Here is the setup:
Arduino UNO R3 http://arduino.cc/en/Main/ArduinoBoardUno
Micro-SD Card Breakout Board+ http://www.adafruit.com/products/254
3-Space Sensor Embedded (3SS) http://tech.yostengineering.com/3-space-sensor/product-family/embedded
SD Library from adafruit.com https://github.com/adafruit/SD
The two SPI devices are connected on the same SPI bus with the Arduino, with each device having its own chip-select (SS) line.

1. Micro-SD Card Breakout Board+ & Arduino:
I have followed the very helpful tutorial provided through adafruit, http://www.ladyada.net/products/microsd/, and can get the Arduino to successfully run all of the example programs provided in the new SD library.

2. 3-Space Sensor Embedded & Arduino:
The 3SS is a complex beast, but I can get it working perfectly with the Arduino SPI library: sending commands and receiving data. It would be best to think of this as a black box, outside of the SPI interface. It operates by receiving commands (from Arduino) and sending data in full duplex. I have this code if necessary.

3. Micro-SD Card Breakout Board+, 3-Space Sensor Embedded & Arduino:
I essentially combined the above two programs into this "end game" program, while making sure the setup remains correct according to each of the requirements. Right off the bat, the program showed itself to be very finicky and prone to stop running in the first loop or having one of the two SPI devices not run/return data. I have tried a myriad of options, hacks and suggestions found through Arduino, adafruit, and other sources; but they are not worth listing unless the topic comes up in the future.

Here is the code:

Code: [Select]

/*
  IMUData2SDcard
  Get the 3-axis accelerometer and 3-axis gyroscope data from the IMU
  using SPI then save the results in a file on the microSD card.
  Uses the SPI and SD library.
 
  Hardware Setup
       Arduino   SDcard   TSS-EM
        pin 7              SS
        pin 10    SS
        pin 11    MOSI     MOSI
        pin 12    MISO     MISO
        pin 13    SCK      SCK
 
  created 03/01/2013
  by David Hughes
*/

#include <SPI.h>
#include <SD.h>

/* Pins used for SPI connection */
const int chipSelectPinIMU = 7;
const int chipSelectPinSDcard = 10;
// the rest are controlled by the SPI library

/* Union for each of the 9 sensors */
// Needed to convert the bytes from SPI to float
union u_types {
    byte b[4];
    float fval;
} data[9];  // Create 9 unions, one for each sensor from the IMU
            //   Gyroscope: x-axis, y-axis, z-axis
            //   Accelerometer: x-axis, y-axis, z-axis
            //   Compass: x-axis, y-axis, z-axis

/* File object */
File myFile;

/* Initialization */
void setup() {
  Serial.begin(9600);
 
  // Start the SPI library:
  digitalWrite(chipSelectPinIMU, HIGH);
  SPI.begin();
  // Set the chip select pin for IMU:
  pinMode(chipSelectPinIMU, OUTPUT);
 
  Serial.print(F("Initializing SD card..."));
  // Set the chip select pin for the SDcard board:
  pinMode(chipSelectPinSDcard, OUTPUT);
  // Initialize the SDcard and library
  if (!SD.begin(chipSelectPinSDcard)) {
    Serial.println(F("initialization failed!"));
  } else {
    Serial.println(F("initialization done."));
  }
 
  // Give the sensor time to set up:
  delay(1000);
}

// Debugging code, to check usage of RAM
// Example Call: Serial.println(freeRam());
//int freeRam () {
//  extern int __heap_start, *__brkval;
//  int v;
//  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
//}

/* Main Loop */
void loop() {
  Serial.print(F("Sending packet..."));
  // Take the chip select low to select the device:
  digitalWrite(chipSelectPinIMU, LOW);
  // Clear the internal data buffer on the IMU
  SPI.transfer(0x01);
  delay(10);
  // Send start of packet:
  SPI.transfer(0xF6);
  delay(1);
  // Send command:
  SPI.transfer(0x40);
  delay(1);
  // Get status of device:
  byte result = SPI.transfer(0xFF);
  while (result != 0x01) {  // Repeat until device is Ready
    delay(1);
    result = SPI.transfer(0xFF);
  }
  // Get the 36 bytes of return data from the device:
  for (int ii=0; ii<9; ii++) {
    for (int jj=0; jj<4; jj++) {
      data[ii].b[jj] = SPI.transfer(0xFF);
      delay(1);
      Serial.print("data[");
      Serial.print(ii);
      Serial.print("].b[");
      Serial.print(jj);
      Serial.print("]: ");
      Serial.println(data[ii].b[jj],HEX);
    }
  }
  // Take the chip select high to de-select:
  digitalWrite(chipSelectPinIMU, HIGH);
  Serial.println(("Packet send/receive complete."));
 
  // Convert Big Endian (IMU) to Little Endian (Arduino)
  for( int mm=0; mm<9; mm++) {
    endianSwap(data[mm].b);
  }
  Serial.println(F("Data endian swap complete."));
 
  // Traverse the union to print the data values
  Serial.println(F("Gyroscope:"));
  for (int kk=0; kk<9; kk++) {
    Serial.print(data[kk].fval);
    if (kk == 2) {
      Serial.println("");
      Serial.println(F("Acceleration:"));
    } else if (kk == 5) {
      Serial.println("");
      Serial.println(F("Compass:"));
    } else {
      Serial.println("");
    }
  }
 
  // Open the file. Note that only one file can be open at a time
  myFile = SD.open("test.txt", FILE_WRITE);
  // Write data to the file
  if (myFile) {  // if the file opened okay, write to it:
    Serial.print(F("Writing to test.txt..."));
    // Traverse the union to print the data values
    myFile.println("Gyroscope:");
    for (int kk=0; kk<9; kk++) {
      myFile.print(data[kk].fval);
      if (kk == 2) {
        myFile.println("");
        myFile.println("Acceleration:");
      } else if (kk == 5) {
        myFile.println("");
        myFile.println("Compass:");
      } else {
        myFile.println("");
      }
    }
    myFile.close();  // close the file
    Serial.println(F("Write to file complete."));
  } else {
    // if the file didn't open, print an error:
    Serial.println(F("error opening test.txt"));
  }
  myFile.println("");
 
  delay(5000);  // Wait x/1000 seconds before next loop
}

/* Endian swap - big to little */
inline void endianSwap(byte temp[4]) {
//  Serial.println("Before swap: ");
//  Serial.println(temp[0],HEX);
//  Serial.println(temp[1],HEX);
//  Serial.println(temp[2],HEX);
//  Serial.println(temp[3],HEX);
  byte myTemp = temp[0];
  temp[0] = temp[3];
  temp[3] = myTemp;
  myTemp = temp[1];
  temp[1] = temp[2];
  temp[2] = myTemp;
//  Serial.println("After swap:  ");
//  Serial.println(temp[0],HEX);
//  Serial.println(temp[1],HEX);
//  Serial.println(temp[2],HEX);
//  Serial.println(temp[3],HEX);
}

Is there a problem using both the Arduino SPI and Arduino SD libraries in a program?
- It seems odd that we could not find any information about the two libraries online, whether good or bad.  We assumed the they worked perfectly well together using the evidence of absence, since decisions have to made for projects with hard deadlines.

I am happy to supply more information, code, pictures, etc.

Thank you!

[SurferTim]

Have you tried them separately with both devices connected? Try the SD sketch with the IMU disabled. Does that run ok?

Code: [Select]

void setup() {
 Serial.begin(9600);
 
 // disable the IMU
 pinMode(chipSelectPinIMU, OUTPUT);
 digitalWrite(chipSelectPinIMU, HIGH);
 
 Serial.print(F("Initializing SD card..."));
 // Set the chip select pin for the SDcard board:
 pinMode(chipSelectPinSDcard, OUTPUT);
 // Initialize the SDcard and library
 if (!SD.begin(chipSelectPinSDcard)) {
   Serial.println(F("failed!"));
 } else {
   Serial.println(F("done."));
 }
}


Then try the IMU with the SD connected but disabled the same way. Does it run ok?

Code: [Select]

 // disable SD
 pinMode(chipSelectPinSDcard, OUTPUT);
 digitalWrite(chipSelectPinSDcard, HIGH);

 // do SPI/IMU setup


[Dave2Satellite]

Good suggestion, thanks for the quick response.

Keeping everything connected:
Disabling the IMU and running the SD works when run, and the "complete" can be seen on the serial window.
Disabling the SD and running the IMU also works perfectly, outputting data in the serial window as expected.

This is a indication of a conflict between the two libraries?

[SurferTim]

That checked hardware compatibility, but I see you figured that out. Now we check library compatibility. Try the combined code with a setup like this:

Code: [Select]

void setup() {
 Serial.begin(9600);
 
 // disable the IMU
 pinMode(chipSelectPinIMU, OUTPUT);
 digitalWrite(chipSelectPinIMU, HIGH);
 
 Serial.print(F("Initializing SD card..."));
 // Set the chip select pin for the SDcard board:
 pinMode(chipSelectPinSDcard, OUTPUT);
 // Initialize the SDcard and library
 if (!SD.begin(chipSelectPinSDcard)) {
   Serial.println(F("failed!"));
 } else {
   Serial.println(F("done."));
 }

 // start SPI
 SPI.begin();
}

edit: If that doesn't work, then try each device sketch with that setup. Basically include the SPI.begin() or the SD.begin() to the other device sketch.

[Dave2Satellite]

The first suggestion, thank you for including the code, allowed the SD card to initialize but the data returned from the 3SS device was mangled. 

Here is a picture of the newly received data:


Here is a picture of what the correct data should look like:


3SS device:
I took the original code and added the following two lines, the first below the include for SPI and the second inside the setup() as suggested.

Code: [Select]

#include <SD.h>
SD.begin();
The data received was also corrupted/mangled when these two lines were added.  I commented out these lines, and the program went back to working correctly.
I did this test a second time, on a more relevant program for our SPI device (i.e. data), and the program would freeze as it got to "wait for command to process" stage.  The 3SS needs to process a sent command, and that can take a few clock cycles, so there is a simple while loop that waits for a return value of 0x01 or "READY".  This is where it fails.  The SPI.transfer(start byte) and SPI.transfer(command) work just fine before this freeze.
NOTE - This is where I started looking into SPI Data Mode, and if there was any conflict between the SPI and SD.  Eventually I gave up, since the code for the SD library has no actual mention of SPI library or its functions.  Does anyone know why the SD library has no mention of the SPI library?  (SD uses SPI to communicate with the SD-card, somehow).

SD-card device:
Again, I took this original code (examples>SD>Files) and added the following two lines, the first below the include for the SD and the second inside the setup() as suggested.

Code: [Select]

#include <SPI.h>
SPI.begin();
Surprisingly, these two lines of code did not affect the successful operation of the program (with SD library functions).  I tried placing the SPI.begin() line in different points in the setup, but it would still work.


I realize this might be not descriptive enough in some areas, so let me know what does not make sense.  As always, communication is critical in this environment.

Thank you again.

[SurferTim]

Try initializing the SD with this in your IMU code (the one with the corrupted data):

Code: [Select]

 Serial.print(F("Initializing SD card..."));
 // Set the chip select pin for the SDcard board:
 pinMode(chipSelectPinSDcard, OUTPUT);
 // Initialize the SDcard and library
 if (!SD.begin(chipSelectPinSDcard)) {
   Serial.println(F("failed!"));
 } else {
   Serial.println(F("done."));
 }

 // add this to check and correct the status of the SD slave select
 if(digitalRead(chipSelectPinSDcard) == LOW) {
   Serial.println(F("SD SS is LOW"));
   digitalWrite(chipSelectPinSDcard, HIGH);
 }

The Ethernet.begin() function leaves the w5100 slave select LOW, and will mess up the SD card. I call Ethernet.localIP() after Ethernet.begin(), and that corrects the slave select.

The standard SD library included with the Arduino IDE does not leave the SD slave select LOW after the begin call, but you are using the SD library from adafruit, correct? That addition to the setup above will check and correct that.

edit: My bad. I forgot digitalRead(). Corrected now.

[Dave2Satellite]

I tried out the code you provided in the IMU code (with the corrupted data) and did not get the "SD SS is LOW" message.  I double checked by adding a line seen here:

Code: [Select]

  Serial.print(F("Initializing SD card..."));
  // Set the chip select pin for the SDcard board:
  pinMode(chipSelectPinSDcard, OUTPUT);
  // Initialize the SDcard and library
  if (!SD.begin(chipSelectPinSDcard)) {
    Serial.println(F("failed!"));
  } else {
    Serial.println(F("done."));
  }
  // add this to check and correct the status of the SD slave select
  if (digitalRead(chipSelectPinSDcard) == LOW) {
    Serial.println(F("SD SS is LOW"));
    digitalWrite(chipSelectPinSDcard, HIGH);
  } else {
    Serial.println(F("SD SS is HIGH, good!"));  // NEW CODE FOR SS HIGH
  }
Yes, we are using the SD library from adafruit. https://github.com/adafruit/SD

Looking at the adafruit SD library, I can see that SD.begin() calls the function card.init().  card is of the class Sd2card.  In the file Sd2Card.cpp the init() function is located at line 270.  Within this function we can see a call to the function chipSelectHigh() in both normal return (line 356) and fail (line 368).  I checked the Arduino SD library for the same functions calls, but there is no call to chipSelectHigh() in a normal return (validating your statement about this SS not being set HIGH). 

The Ethernet.begin() function leaves the w5100 slave select LOW, and will mess up the SD card. I call Ethernet.localIP() after Ethernet.begin(), and that corrects the slave select.

I was not sure about this part, but I think you were showing an example of how to fix the SS by calling a function that sets it HIGH.

[SurferTim]

I was not sure about this part, but I think you were showing an example of how to fix the SS by calling a function that sets it HIGH.

Yes. If you see "SD SS is LOW" in the serial monitor, you will know the SD.begin() function from adafruit does not disable the slave select, and will correct that IF the slave select was LOW.

[Dave2Satellite]

The serial does not output "SD SS is LOW".  Or, it will output "SD SS is HIGH, good!" if you use the code I modified in my last post.

So we know that the initialization for the SPI and the SD are correct, only in terms of the SS being set HIGH after each device is initialized.  Is it safe to say the problem lies inside the begin calls for SD and SPI?  (I based this on our recent findings and that the SD.begin() will fail and print "failed!" to the serial output).

What are your thoughts?

[SurferTim]

I use the standard SD library included with the Arduino IDE, so I do not have experience with the adafruit library. I know the IDE SD library does not interfere with other SPI devices. I use it on a regular basis with the w5100 without fail. The w5100 library uses SPI.begin() to set up the SPI.

[Dave2Satellite]

The standard SD library included with the Arduino IDE does allow the SD.begin() to return true, but we can still see the same data problems over the SPI with the IMU device.  Here is the initialization code:

Code: [Select]

/* Initialization */
void setup() {
  Serial.begin(9600);
  // start the SPI library:
  SPI.begin();
  // initalize the chip select pin:
  pinMode(chipSelectPin, OUTPUT);
 
  // Code from Arduino Forum, by SurferTim
  if (digitalRead(chipSelectPin) == LOW) {
    Serial.println(F("SPI SS is LOW"));
    digitalWrite(chipSelectPin, HIGH);
  } else {
    Serial.println(F("SPI SS is HIGH, good!"));
  }
 
  // Code from Arduino Forum, by SurferTim
  Serial.print(F("Initializing SD card..."));
  // Set the chip select pin for the SDcard board:
  pinMode(chipSelectPinSDcard, OUTPUT);
  // Initialize the SDcard and library
//  if (!SD.begin(chipSelectPinSDcard, 11, 12, 13)) {
  if (!SD.begin(chipSelectPinSDcard)) {
    Serial.println(F("failed!"));
  } else {
    Serial.println(F("done."));
  }
  // add this to check and correct the status of the SD slave select
  if (digitalRead(chipSelectPinSDcard) == LOW) {
    Serial.println(F("SD SS is LOW"));
    digitalWrite(chipSelectPinSDcard, HIGH);
  } else {
    Serial.println(F("SD SS is HIGH, good!"));
  }
  // End code by SurferTim
 
  // Give the sensor time to set up:
  delay(100);
}
We are desperate for a solution to this problem, do you have any concrete examples of the two working together?  I do not know what a w5100 is or know anything about the 5100 library, but would assume the code is similar in these circumstances.

Thank you again,
Dave

[SurferTim]

Hi Dave. I only have one more suggestion for you. One of the things the SD.begin() function apparently does is double the speed of the SPI bus. After the SD.begin call, add this as a test:

Code: [Select]

SPI.setClockDivider(SPI_CLOCK_DIV8);


[Dave2Satellite]

I added your line of code after the SD.Begin() call, but we still see the same problem with interference of the SPI data being returned from the IMU. 

[SurferTim]

OK, one more thing. Are you sure you are not running out of memory? I use this freeRam() function to check available SRAM. If you run out, it does weird things. Check your IMU sketch without the SD.begin() call, and see how much memory it shows. Then use SD.begin() and check.

Code: [Select]

int freeRam() {
  extern int __heap_start,*__brkval;
  int v;
  return (int)&v - (__brkval == 0 ? (int)&__heap_start : (int) __brkval); 
}

void setup() {
  Serial.begin(9600);
  Serial.print(F("Free SRAM = "));
  Serial.println(freeRam());
}

void loop() {
}
It will not show zero if out of memory. It will show either a negative number, or an unrealistic positive amount.

[Dave2Satellite]

I did the check throughout the code with the SD include and SD.begin() and got a value around 1050.  I took out the SD library and the SD.begin() lines and got around 1660.  These do fit within the available SRAM for the Arduino and I did not see anything negative or funky.

After speaking with Mr. Greiman, the creator of the SdFat library which the SD is based on, I am going to scrap using the SD library from Arduino and focus on using the SdFat library instead.  We are still open to any suggestions or questions, but time is short and the decision had to be made in which direction to head to complete the project within the time frame.  

For those looking for answers on this topic, we found a solution to the problem!

Using the SdFat library solved all of our problems, and it only took a few modifications to the original programs.  The SdFat library is the original basis of the SD library, but the SD library is based on an old/outdated version of the SdFat library.  I want to thank Bill Greiman for pointing me in the right direction!

Here is a link to the SdFat download http://code.google.com/p/sdfatlib/downloads/list

Thank you very much for your time and help SurferTime, we are just happy to have a solution.  (We learned a lot during your suggestions/help too!)

Best,
Dave