Reading audio from wav file on SD card

Is there a function in SD.h or SdFat.h that can tell me how many bytes are in the library's input buffer? Or at least tell me if there are any? My understanding is that available() tells me how many bytes are left to read in the file. But basically, I want to know whether a read() would immediately produce a byte, or would be blocking because at the moment the buffer is empty.

Also, if the buffer is empty, does the library automatically try to fill it if there are bytes remaining in the file, or would it wait until I read() to do that?

I'm asking because I will need to read an audio file from an SD card and play it. It will be a wav file sampled at 8Ksps, and it will be unsigned 8-bit , perfect for writing to an R-2R ladder on Port D. I don't want there to be any delays or dropouts, but I don't know if 8KB per second from an SD card is ambitious for an Arduino 328P.

Neither SD.h nor SdFat provide a way to inspect the internal buffer. A read will trigger fetching the next 512 buffer once you’ve read everything from the current bufffer.

As SdFat is faster and more efficient than SD.h, I think it’s a better choice for your project On an ATmega328P, reading at 8 kB/s (8 kHz, 8-bit) should be within reach, because a 512-byte sector can be read in under 1 ms, while you need a new sector only every ~64 ms.

To output samples without dropouts during a fetch, you should use a double buffer in RAM: one buffer is read from the SD card in the main loop while the other is fed to the DAC (your R-2R ladder) in a timer interrupt, ensuring precise timing for audio playback.

Consider this TMR20h audio library: Auto Analog Audio: Automatic Analog Audio Library for Arduino and/or the TMRpcm library GitHub - TMRh20/TMRpcm: Arduino library for asynchronous playback of PCM/WAV files direct from SD card. Arduino Uno,Nano,Mega etc supported

I was planning to use a 256-byte circular buffer. The loop() would try to keep it filled, and the ISR would empty it. I think something like that should work as long as the interrupts can continue to take place during SPI/SD activity.

I guess I should look at that. It just seemed complicated compared to simply writing each byte to port D. And the R-2R ladder reportedly produces good sound.

So do the TMR libraries, without possible complications from incorrect ladder resistor values, as mentioned in your other post.

Worth trying

More questions about reading wav files from an SD card.

Google's AI tells me that when an Arduino SD library reads the next 512-byte sector, that entire process is a blocking event. So loop() activity is frozen during that event. I don't know if interrupts are also disabled.

But it also says that's not necessary:

"When you issue a read command to the SD card, the card begins to stream data over the SPI bus. It does not enforce a strict time limit on the total time it takes to transmit the entire block. The protocol is designed to be tolerant of pauses by the host, since the host microcontroller may be busy with other tasks. The card's controller will wait for the host to send the clock pulses required to read the next byte."

So it seems it should be possible for the library to essentially read one byte at a time in a process that would not be blocking. Is there an SD library that does that? I found @fat16lib's SdBoot, which it looks like might do that. But otherwise, what do SD.h and SdFat.h do? Is the AI right about them being blocking?

Another problem with my current project is that there's a lot going on, and I need to get the flash usage for this part down. I wonder if it's read-only, and I adopt the restriction that the audio file's sectors be consecutive, so it doesn't really have to consult the FAT table at all for the next cluster, then the SD code might be fairly small. It would need to provide for SD and SDHC, but possibly not SDXC or larger.

I have no experience with TMRpcm.h, and don't know whether it is able to keep up. It's almost 2000 lines of code, and I haven't been able to find how it actually handles the SD reading part.

Well, if anyone is knowledgeable about all this, i would appreciate any clarifications or suggestions.

A quick glance at the TMRpcm library, specifically the file TMRpcm.h, reveals that it uses the SdFat library to access the SD card:


#ifndef TMRpcm_h   // if x.h hasn't been included yet...
#define TMRpcm_h   //   #define this so the compiler knows it has been included

#include <Arduino.h>
#include <pcmConfig.h>
#include <pcmRF.h>
#if !defined (SDFAT)
    #include <SD.h>
#else
    #include <SdFat.h>
#endif

By the way, the SD library is a simplified approach intended for Arduino programmers, and uses the SdFat library to do the actual work. I haven't looked to see the reason for the choice to use either in TMRpcm.

I ended up using two timers to generate the audio output. Timer2 is set to run phase-correct PWM at about 32KHz with output on D3, and timer1 just generates an interrupt 8000 times per second. The ISR loads a byte from the circular buffer and stores it in OCR2A.

But I have a problem trying to determine how well the circular buffer keeps up to speed. My code calculates how full the buffer is, and keeps track of the lowest fill state during the entire file read. But with a 512-byte buffer, I keep getting 255 as the smallest buffer fill. That's a problem because when I switch to a 256-byte buffer, I get 217 as the smallest buffer state, and there's no reason a larger buffer should perform worse than a smaller one. Also, 255 kinda suggests that I've messed up the unsigned subtraction. But I can't find what's wrong.

The code that keeps track of the smallest buffer fill is shown below. "Max" is the buffer size minus 1, so 0x1FF for the 512-byte buffer.

  while (file.available()) {

    bufState = (head - tail) & Max;
    if (bufState < smallestState) smallestState = bufState;

    nexthead = (head + 1) & Max;
    if (nexthead != tail) {
      buf[nexthead] = file.read();
      head = nexthead;
    }
  }
  while (head != tail);                  // wait until buffer emptied by ISR

And here's the whole sketch. Maybe somebody can see where I've gone wrong.

// for 328P
// 10 CS, 11 MOSI, 12 MISO, 13 SCK
// audio out D3, signal LED D6

#include <SPI.h>
#include <SD.h>

File file;

const int signalLED = 6;
const unsigned int bufsize = 512, Max = bufsize - 1;  // size must be power of 2
volatile byte buf[bufsize];
volatile unsigned int head, tail, nexthead;
unsigned int i, j, bufState, smallestState;
bool match;

void setup() {
  Serial.begin(9600);
  digitalWrite(signalLED, LOW);
  pinMode(signalLED, OUTPUT);
  Serial.println("start");
  delay(3000);

  if (!SD.begin(10)) exit();
  file = SD.open("440wav.wav");          // 8-bit samples, 8000 samples per second, mono
  if (!file) exit();
  SPCR &= 0xFC;                          // SPI speed = 4MHz

  for (i = 0; i < bufsize; i++) {        // fill buffer from file
    if(file.available()) {
      buf[i] = file.read();
    }
    else {
      i++;
      break;
    }
  }
  head = i - 1;                          // head points to last byte added to buffer from SD
                                         // tail points to last byte fetched by ISR
                                         // if head+1 == tail, the buffer is full
                                         // if head == tail, the buffer is empty
  match = false;
  for (j = 0; j < (bufsize - 7); j++) {
    if ((buf[j] & 0xDF) != 'D') continue;
    if ((buf[j+1] & 0xDF) != 'A') continue;
    if ((buf[j+2] & 0xDF) != 'T') continue;
    if ((buf[j+3] & 0xDF) != 'A') continue;
    match = true;
    break;
  }
  if (match) tail = j + 7;               // set tail to last byte before audio
  else exit();
  Serial.println(head);
  Serial.println(tail);
  Serial.println();
  delay(100);

  while (file.available()) {             // refill buffer
    nexthead = (head + 1) & Max;
    if (nexthead != tail) {
      buf[nexthead] = file.read();
      head = nexthead;
    }
    else break;
  }
  smallestState = (head - tail) & Max;   // print beginning smallestState
  Serial.println(smallestState);
  Serial.println();
  delay(100);

  noInterrupts();                        // disable all interrupts

  TCCR1A = 0;                            // set up Timer1 on Mode 4 - CTC,
  TCCR1B = 0;                            //   at exactly 8000 interrupts/sec
  TCNT1  = 0;
  TCCR1A = 0;                            // CTC mode, top = OCR1A, no output
  TCCR1B = (1 << CS11) + (1 << WGM12);   // CLK/8
  OCR1A = 249;                           // 16mil / 8 / 250 = 8000 interrupts/sec
  TIMSK1 = 0;                            // interrupt disabled for now
  TIFR1 = 0x27;                          //    and flags cleared by writing 1

  TCCR2A = 0;                            // set up Timer2 on phase-correct PWM,
  TCCR2B = 0;                            //    31373HKz, output on D3
  TCNT2  = 0;
  TCCR2A = (1 << COM2B1) + (1 << WGM20); // non inverting / 8Bit PWM
  TCCR2B = 1 << CS20;                    // CLK/1
  OCR2B = 128;                           // 128 is the center ("0") of the waveform
  TIMSK2 = 0;                            // interrupt disabled
  TIFR2 = 7;                             //    and flags cleared by writing 1
  pinMode (3, OUTPUT);                   // allow Timer2 to drive D3

  interrupts();                          // enable all interrupts

  TIMSK0 &= 0xFE;                        // disable timer0 overflow interrupt - millis
  TIMSK1 = 1 << OCIE1A;                  // TC1 compare match Interrupt Enable
                                         //   (no interrupts from TC2)

  while (file.available()) {

    bufState = (head - tail) & Max;
    if (bufState < smallestState) smallestState = bufState;

    nexthead = (head + 1) & Max;
    if (nexthead != tail) {
      buf[nexthead] = file.read();
      head = nexthead;
    }
  }
  while (head != tail);                  // wait until buffer emptied by ISR

  TIMSK1 = 0;                            // disable timer1 interrupt
  TIMSK0 |= 1;                           // enable timer0 overflow interrupt - millis

  pinMode(3, INPUT);                     // disconnect timer2 from D3
  TCCR2A = 0;                            // turn off timer2
  TCCR2B = 0;
  TCCR1A = 0;
  TCCR1B = 0;

  file.close();
  digitalWrite(signalLED,HIGH);
  Serial.println(smallestState);
}

void exit() {
  digitalWrite(signalLED, HIGH);
  while (1);
}

void loop() {
}


//**************************************************************************
// Timer1 interrupt at 8 KHz puts next audio sample into Timer2 OCR2B
//**************************************************************************
ISR(TIMER1_COMPA_vect) {

  if (head != tail) {                    // if buffer not empty, get next byte
    tail = (tail + 1) & Max;
    OCR2B = buf[tail];
    digitalWrite(signalLED,LOW);
  }
  else digitalWrite(signalLED,HIGH);     // LED lights if buffer empty
}

You have

And

I have not read everything but This sounds risky.

If head == tail buffer is empty and you do get 0 and if you have a full buffer then head - tail is 512 and if you mask it with 511 then you get 0 too.

Do you let the buffer go full? If you do then it’s an issue as you’ll think the buffer is actually empty

A simple workaround would be to do

(head - tail + bufsize) & Max;

Also The ISR runs asynchronously with the main loop and modifies tail, while the main loop computes (head - tail) & Max.

Because head and tail are multi-byte values, an interrupt can fire in the middle of reading or writing them, producing inconsistent results.

That could also explain why the buffer state looks wrong.

The safe way is to make that atomic and protect accesses to shared variables with noInterrupts()/interrupts() (or cli()/sei()) around those calculations, so the ISR can’t interfere during the read–modify–write sequence or make a copy in a critical section and use the copy in the loop.

Thanks for the reply.

If the buffer is full, the head and tail will be one location apart, so the difference would be 511 at most, and ANDed with Max(511) it is still 511. If head and tail are the same, that means the buffer is empty, so the difference will be zero to start with.

However,

I think this will be the explanation. It would be the MS byte being messed up as the tail pointer rolls through zero. And that would also explain why I don't get the problem with a 256-byte buffer using the exact same code - the MS byte is always zero. Thanks again.

Film at 11.

Ok yes make sense for 511.
(I did not look much into the code)

The atomicity of operations is indeed something to consider

Ok, @J-M-L, you were right about the interrupts messing up the two-byte values. I added a line to refrain from doing anything when timer1 is about to generate the interrupt:

  while (file.available()) {

    if (TCNT1L == 249) continue;
          
    bufState = (head - tail) & Max;
    if (bufState < smallestState) smallestState = bufState;

    nexthead = (head + 1) & Max;
    if (nexthead != tail) {
      buf[nexthead] = file.read();
      head = nexthead;
    }
  }
  while (head != tail);                  // wait until buffer emptied by ISR

So now using SD.h and the 256-byte buffer, I get 217 as the lowest value. The 512-byte buffer produces 473. The drop is the same in both cases, as I would have expected.

Using SdFat.h however, I get 238 and 494. Apparently SdFat.h reads the card a bit more efficiently, and never falls quite as far behind keeping the buffer full. But if this performance is typical, there would be no need for more than 256, and maybe 128 would work as well. But in this case the file was saved to a freshly formatted card, so all the clusters are consecutive. If a file were to be fragmented, I don't know how much things would be affected.

But SdFat now uses a bit more flash than SD, and that may matter. I've been thinking about all the stuff this project will include, and I may have to hunt for a much smaller SD library. This sketch alone takes up 37% of total flash, but I really only need to read the file, so maybe there's a smaller alternative.

Anyway, thanks for bailing me out on this. I don't know if it would ever have dawned on me.

Glad you got it to work !

And if it’s 250 when you get there ?

249 is as high as it goes. Supposedly when it reaches 249, it goes to zero and generates an interrupt. But the question is whether it does that immediately, or only on the next clock cycle - what would have been the 250 clock. So as far as I can tell, it stays at 249 for a full clock tick, which is 125uS. It might be safer to hold off when it's 248 or 249, but just 249 seems to work fine.

I guess the normal method would be to use CLI/SEI, but I don't want to affect the timing of any of the interrupts. Of course one answer is to just use a 256-byte buffer, which I may have to end up doing anyway.

It looks like I may not have enough flash or ram in the Nano for this project. I'll have Wire, SPI, SD, DS3231, EEPROM, 16x2 LCD display, rotary encoder, and DTMF generating and decoding, plus Menu stuff. To have a chance, I think I'm going to need a really small, read-only SD library. Back in the day, I wrote an SD bootloader for a couple MSP430 parts which was read-only, and it fit into 1K of flash. But it was written in MSP430 assembler. I would hate to have to convert it to Arduino speak, but that might be the answer. It only worked for SD and SDHC, and only for files stored in consecutive sectors (no FAT traversing needed), but it was fast! I was kinda hoping someone had already done that for Arudino, but not that I can find. This diagram shows why talking to an SD card isn't easy:

I found a small SD library. It's Bill Greiman's (@fat16lib) SdBoot:

https://forum.arduino.cc/t/small-sd-card-code/87554/18

A sketch to read in the file used 2764 bytes of flash, so very much smaller than SD.h. But it also used 797 bytes (38%) of ram, so further work to do. But this is encouraging. Thanks, Bill.

If you want to stream a wave file from an SD to a AVR Arduino, you may want to look at this library. I wrote it in 2008-2009 and Adafruit has maintained it till recently.

It use a special SD reader that reads half sectors and streams the file to an DAC in a interrupt routine controlled by a timer.

Thanks very much. I know from my experience with my SD bootloader for the MSP430 that you don't have to have a 512-byte buffer for SD reading. One of those chips only has 256 bytes of ram in total, and the bootloader works just fine. So I was planning to modify your SdBoot to use a smaller buffer, and to make it a circular buffer when reading in the data. And as you said, bytes are taken out of the buffer by a timer ISR at 8000 or 16000 bytes per second. But I will take a look at WaveHC to see if it may work better. I just don't want two buffers. There's no need for that.

My bootloader required that the file be continuous, and did not mess with the FAT at all. In some cases that was easy because the file being loaded would fit into one cluster. But for a WAV file, I'm wondering whether I should save more space by requiring that the file be contiguous, and eliminating all the cluster code entirely. That would also prevent any related slowdowns, so the buffer could be even smaller.

I really appreciate all your work on SD.