Hi,
I have a program that reads an sd card using sd fat, then outputs the data through an external DAC. I am noticing that my output stalls for around 3ms while another block is read from the sd card. The file on the sd card is a binary file around 1.3MB, it contains DAC values for 3 waveforms. I can see on my scope that the waveforms aren’t continuos because I can see a flat spot where the loop has stopped to read more data into the sd library buffer. The sample rate of each waveform is 6.4 kHz of 16 bit samples. So all threes waves together would add up to 19.2 kHz of 16 bit samples, technically. I can see lots of posts of people using the Arduino to play 44.1khz 16 bit mono tracks, so I am wondering how do They output the music file from the sd card at a continuous rate without their program slowing down to read another 512 byte block from the sd card? Any help is much appreciated thank you.
I have a program
Let's see it.
There are several examples in the SdFat library that can measure the performance of your card. 38kB/s is not that hard.
I had the same problem playing music.. This example uses a ring buffer. The player puts out a byte at each interrupt while the main loop fills the buffer with new data as required.
I used Audacity to save mu sound files as 8bit raw pcm.
(I hope some norwegian words wont be too problematic)
/*
8bits pcm raw sound via shiftregister 595
CS = RCK - to digital pin 9 (SS pin-> portB.1)
CS (SD-kort) - to digital pin 10 (SS pin-> portB.2)
SDI=SI - to digital pin 11 (MOSI pin)
MISO=DI - to digital pin 12 (MISO pin)
CLK=SCK - to digital pin 13 (SCK pin)
*/
#include <SD.h>
// portD.7 (Datapin 7) (HIGH-LOW to output latched data)
// portB.0 (datapin 8) for dataoutput to shiftregister (74HC595)
// portB.1 (Datapin 9) (pulse HIGH -LOW to clock data into register
#define SDcs 10 // chipselect for SD-card
byte sample; // sound sampeled at 8kHz
volatile int tcnt2 = 7;
volatile boolean ready;
volatile int bmax = 200;
volatile byte ring[200]; //index 0..100
volatile int fillpos,playpos,lastbyte;
File dataFile;
long qq;
void setup()
{
pinMode (7, OUTPUT); // sr - RCK
pinMode (8, OUTPUT); // sr - data in
pinMode (9, OUTPUT); // sr - data CLK
pinMode (SDcs, OUTPUT); // SS for SD-card
SD.begin(SDcs);
Serial.begin(115200);
}
void playit(byte hastighet) // hastighet = 8,11 or 16KHz
{
cli();
lastbyte=bmax; // flag to indicate point to stop music if <bmax
fillpos=2 ; // start just ahead of player
playpos=0 ;
switch (hastighet)
{
case 11: tcnt2=78; break; // 11025
case 16: tcnt2=136; break; //16000
default: tcnt2=7; // 8000
}
ready=false; // ISR to set to true 8000 times/sek (or 11000 or 16000)
// setup interrups every 1/8000 sec(if normal speed) sec for playback timing
TIMSK2 &= ~(1<<TOIE2); // Disables ALL interrupts for timer2 (OCIE2B,OCIE2A and TOIE2)
// internal/external clock - does not matter
// ASSR |= (1<<AS2); or ASSR &= ~(1<<AS2);
// Config timer2 counting - normal. Clear bits WGM20,WGM21 and WGM22
TCCR2A &= ~((1<<WGM21) | (1<<WGM20));
TCCR2B &= ~(1<<WGM22);
// conf. prescaler to 16MHz/8
TCCR2B |= (1<<CS21); // Set 1 bit
TCCR2B &= ~((1<<CS20) | (1<<CS22)); // Clear 2 bits
TCNT2 = tcnt2;
TIMSK2 |= (1<<TOIE2); // enable ovf.intr. in mask
// dataFile.seek(0); // start at beginning of the current, open, file
//if (ready==true) // play next byte; never to catch up with reading from file
// play this file till end
sei(); // allow interrups for playback
//************* READ FILE INTO RING-BUFFER ***************
while (dataFile.available())
{
if (fillpos!=playpos) // start...fill...play...max
{ ring[fillpos]=dataFile.read(); // read 'next' byte
fillpos+=1;
if (fillpos>=bmax) fillpos=0;
}
} // while_loop until whole file is read into buffer.
// eof
lastbyte=fillpos-1;
if (lastbyte<0) lastbyte=0; // just in case...
}
//*************** ISR will play notes from buffer ************
ISR(TIMER2_OVF_vect) // runs every 0.125ms ~ 8000Hz , normal speed
{
TCNT2 = tcnt2;// reload timer value
// if (playpos!=fillpos)
{
// play this tone from buffer(playpos)
byte note=ring[playpos];
byte mask=128; // start masking MSB;
for (byte k=0; k<8; k++) // 8bits..
{
// databit to pin 8 ~ portB.0 -> Shiftegister data_in
if ((note&mask)>0) PORTB|=0x01; else PORTB &=0xFE;
// send clk_puls on pin 9 ~ PORTB.1 -> shiftreg. CLK
PORTB |= 0x02; PORTB &= 0xFD; //short clk_pulse
mask>>=1; // prepare mask for 'next' bit
}
// this byte now transmitted -> play this byte
PORTD |= 0x80; PORTD &= 0x7F; //short rck_pulse
// set up next position to play
playpos+=1;
if (playpos>=bmax) playpos=0;// past buffer size
if (playpos==lastbyte) TIMSK2 = 0; // disable timer2 interrups = stop playing
}
qq++;
ready=true;
}
// end of ISR
//**** prog. Opens file -> calls 'playit()' -> close file
void loop()
{
dataFile = SD.open("MARIO1.raw");
playit(11); // speed: 8 eq 8KHz, 11 eq 11,025KHz, 16 eq 16KHz
dataFile.close();
dataFile = SD.open("letter.raw");
playit(16); // speed: 8 eq 8KHz, 11 eq 11,025KHz, 16 eq 16KHz
dataFile.close();
dataFile = SD.open("aero.raw");
playit(11); // speed: 8 eq 8KHz, 11 eq 11,025KHz, 16 eq 16KHz
dataFile.close();
dataFile = SD.open("ant-mary.raw");
playit(11); // speed: 8 eq 8KHz, 11 eq 11,025KHz, 16 eq 16KHz
dataFile.close();
dataFile = SD.open("ballons.raw");
playit(11); // speed: 8 eq 8KHz, 11 eq 11,025KHz, 16 eq 16KHz
dataFile.close();
}
This is my code so far. I am using the Arduino Due. I have tried the Sdfat speed test multiple times over the last few weeks, I can read data at 380 KB/s using my sd card setup. I dont think the read speed is the issue though, i believe its the opening of the file to grab another 512 bytes for the library buffer that is introducing a lag in my output. Thats just a guess though, I could be completely wrong! Any idea how I could solve this problem? I have read about a scheduler for the Due, but Im not sure exactly how that works though, or if it would be suitable for this project.
void outputWave(char WaveFormFile[] ) {
char line[18];
char line1[5];
char line2[5];
char line3[5];
int one=0;
int two=0;
int three=0;
uint16_t red=0;
uint16_t yellow=0;
uint16_t blue=0;
int n;
// open test file
SdFile rdfile(WaveFormFile O_READ);
// check for open error
if (!rdfile.isOpen()) {
Serial.println("error opening file");
}
// read lines from the file
while ((n = rdfile.fgets(line, sizeof(line))) > 0) {
if (line[n - 1] == '\n') {
strcpy(line1, strtok(line , ","));
strcpy(line2, strtok(NULL, ","));
strcpy(line3 , strtok(NULL, " "));
one=strtol(line1, NULL, 16);
two=strtol(line2, NULL, 16);
three=strtol(line3, NULL, 16);
// lineOne=(uint16_t)one;
// lineTwo=(uint16_t)two;
// lineThree=(uint16_t)three;
dataWrite(ssDAC1,writeCh1,one);
dataWrite(ssDAC1,writeCh2,two);
dataWrite(ssDAC1,writeCh3,three);
digitalWriteDirect(ldacPin,0);
digitalWriteDirect(ldacPin,1);
}
}
}
You wont get rid of the card page-shift delay.. I suggest an interruptdriven solution.. and a ringbuffer
Thank you for the code Knut_ny, I think using an interrupt to output the data to the DACwould be an excellent idea. I am wondering though how your output doesnt still lag every so often from reading the blocks of the SD card when you update the buffer. I can see your interrupt is running at 0.125ms, I am wondering how you are able to update your buffer in less than 0.125ms before the next interrupt occurs?Could you explain how your ring buffer works in a bit of detail? Thank you again by the way, it is definetely getting me thinking!
That's the idea with interrups.. It ticks along, even while the main loop is delayed
by a slow SD-transfer.
Cool. My next question is this, my interrupt will output the data to the external DAC every 156 microseconds via SPI. My sd card is also on the SPI bus. What happens if my interrupt happens when the buffer is updating from the SD card? Will it damage the sd card? Or will it go back and continue where it left off in the current 512 byte block in the sd card? I have some pseudo code written below, would this work?
void ISR outputToDac(){
This interrupt outputs 3 bytes via SPI to the external DAC;
}
setup{
create 6.4khz timer.
attach interrupt to a timer that will run at 6.4khz;( 156usecs).
}
loop(){
open binary file on the sd card;
while(1){
Read sd card for 1024 bytes into first buffer;
Turn on interrupt routine above, interrupt now starts outputting each element in the buffer every
156 usecs.
When buffer is half empty, call sd card and fill second buffer;
Start reading from second buffer after first is empty.
When second buffer is half empty read from the card again and fill first buffer.
Repeat until file is empty.
}
}
Would anyone have any real code laying around that could implement this? I would also only like to have the interrupt working when ever the file is opened, is it possible to do that and then turn it off when the file is closed? Any hep is much appreciated!
My sd card is also on the SPI bus.
I would suggest that you don't do this. In the Adafruit wav shield they bit bang an SPI interface because the act of disabling the SD card to let the D/A have a go causes delays when opening the file again.
What happens if my interrupt happens when the buffer is updating from the SD card?
As the SPI interface is hardware then interrupts do not disrupt the process.
Hi Grumpy_Mike, thank you for your input. I dont have an Adafruit wav shield, I am only using the Due and a few quad bipolar SPI DACs. I have already built up a good project for school, it is a project that can create 8 simultaneous sine waves that can be any phase angle or amplitude, I am using sine look up tables that are saved on flash memory. I have added the sd card playback option as a bonus to the project, I can read the file and output the data to the DACs but on the oscilloscope I can see flat spots on my sine waves were my program stops to update a buffer. I would like to solve this hopefully by writing better code, hopefully with interrpts, as I dont really have the option of adding extra hardware. Are there any possible solutions if my sd cards are DAC continue to be on the same SPI bus?
on the oscilloscope I can see flat spots on my sine waves were my program stops to update a buffer.
So you need a bigger buffer for the SD card data, how big is it now?
I dont actually know for sure Mike, I have uploaded my code a few posts up if you dont mind taking a look at it. I am using the sdfat library, i have used some example code from the library that reads grabs one line at a time. I could be completely wrong but i feel that there is a buffer somewhere behind the scenes in the sdfat library that is 512 bytes big. To be honest Mike my coding skills are limited, but I have been learning very quickly over the last few months thanks to people like yourself, could you talk me through a possible option if you know one?
Do you mean the code in reply #3?
To be honest it looks a bit of a mess. You seem to be reading strings out of the SD card! That is not the way to do things at all, you should be reading bytes out of the card. You have not got enough time to dick about with strings and how did you put CR and LF characters in sample code where the samples could be anything? And I can’t see how these strings get converted into samples.
Also you have only posted one function that calls other functions that we do not know about, you should post all your code so we can try it out.
Sorry for the confusion but I will try and explain the code above-
So this code is a couple of weeks old now, I had an excel file with 3 columns of data, each column of data represented a sine wave with varying amplitude and frequency througout the file, the sample rate was 6.4kHz, so it has 128 samples for every full cycle of the sine wave. There were 224000 rows of data, roughly 35 seconds of data. I want to be able to play this data again through a DAC. I mapped all of the values to a range between 0 and 65535, then I used the DEC2HEX function in excel to convert each number to a 4 character HEX representation. I then save thos data as a CSV text file. So now I had a file that looked like this
F123,DBEC,BEC3
A2D3,BEC4,FFFF
etc.......
The reason I did this was to keep each line the same size, so I would know that each line read would take the same time, and this would help me get a constant frequency when trying to read the file. I found the sdfat library online and hooked up my sd card, I ran the sdfat bench test and I am able to read at 660KB/s. I also took some code from an sdfat example in the library that reads an sd card one line at a time, and that is what I am using in my code so far. So the process of the code is this-
1.A nextion library listens for a command from the touch screen. When a button is pressed it changes 2 states called playbackModeState & outputWaveTest, when these are true then the program goes to the loop where the program starts play back of file on sd card.
2.open a csv text file of HEX values.
3.read line from file and save it in the char array called 'line'.
4.Parse this data into 3 seperate char arrays line1, line2 and line3, using comma as delimiter.
5.Use strtol to convert char arrays into integers called one, two and three.
6.Then send the integers one, two and three with appropriate commands and addresses to an external DAC via SPI transfrers using the function dataWrite().
7.The function digitalWriteDirect() turns a pin low, then back high again, this signals the external DAC to update its register after the data has been loaded.
The code is working to an extent, I get an output on my DAC, and using an ocsilloscope I can see the waveforms, however there seems to be a lag in the code, I have attached a picture of my output. The picture shows flat spots on the 3 waveforms, the flatspots all happen at the same time, which leads me to believe there may be a buffer updating somewhere in the sdfat rdfile.fgets function. My program has quite a few lines so I will try and make this a bit easier and only post the code concerning the sd card playback.
#include <BlockDriver.h>
#include <FreeStack.h>
#include <MinimumSerial.h>
#include <SdFat.h>
#include <SdFatConfig.h>
#include <SysCall.h>
#include <SPI.h>
#include <Nextion.h>
#include <string.h>
#include <sstream>
#include <stdlib.h>
const uint8_t chipSelect = 52;
SdFat sd;
#define error(s) sd.errorHalt(F(s))
int clrPin = 8; int ldacPin = 9; int ssDAC1 = 4; int ssDAC2=10;
NexTouch *nex_listen_list[]={&bt0,&bt1,&MMB,&PMB,&BMB,&FUB,&FDB,&BPM,&PMU,NULL};
char waveTEST[]="testCSVHEXtxt.txt";
char line[18];
char line1[5];
char line2[5];
char line3[5];
int one=0;
int two=0;
int three=0;
uint16_t red=0;
uint16_t yellow=0;
uint16_t blue=0;
int n;
void setup() {
Serial.begin(115200);
nexInit();
SPI.begin(ssDAC1);
SPI.begin(ssDAC2);
SPI.setClockDivider(ssDAC1, 4);
SPI.setClockDivider(ssDAC2, 4);
SPI.setBitOrder(MSBFIRST);
if (!sd.begin(chipSelect, SPI_FULL_SPEED)) sd.initErrorHalt();
Serial.println("set up complete");
delay(1000);
}
void loop() {
nexLoop(nex_listen_list);
while(playbackModeState==1 && outputWaveTest==1){
outputWave(waveTEST);
outputWaveTest=!outputWaveTest;}
}
void dataWrite(int ssDAC, byte command, uint16_t data) {
byte b2 = data >>8;
byte b3 = data;
SPI.transfer(ssDAC, command,SPI_CONTINUE);
SPI.transfer(ssDAC, b2,SPI_CONTINUE);
SPI.transfer(ssDAC, b3,SPI_LAST);
}
static inline void digitalWriteDirect(int pin, boolean val){
if(val) g_APinDescription[pin].pPort -> PIO_SODR = g_APinDescription[pin].ulPin;
else g_APinDescription[pin].pPort -> PIO_CODR = g_APinDescription[pin].ulPin;
}
void outputWave(char WaveFormFile[] ) {
// open test file
SdFile rdfile(WaveFormFile, O_READ);
// check for open error
if (!rdfile.isOpen()) {
Serial.println("error opening file");
}
// read lines from the file
while ((n = rdfile.fgets(line, sizeof(line))) > 0) {
if (line[n - 1] == '\n') {
strcpy(line1, strtok(line , ","));
strcpy(line2, strtok(NULL, ","));
strcpy(line3 , strtok(NULL, " "));
one=strtol(line1, NULL, 16);
two=strtol(line2, NULL, 16);
three=strtol(line3, NULL, 16);
// lineOne=(uint16_t)one;
// lineTwo=(uint16_t)two;
// lineThree=(uint16_t)three;
dataWrite(ssDAC1,writeCh1,one);
dataWrite(ssDAC1,writeCh2,two);
dataWrite(ssDAC1,writeCh3,three);
digitalWriteDirect(ldacPin,0);
digitalWriteDirect(ldacPin,1);
}
}
}
I know that creating a hex csv file for 16 bit samples is ludicrous. So recently I have been able to convert all of my values into a binary file that is just filled with the 16 bit samples one after another using matlab.
So now my new file is like this,
wave1 uint16 bit sample, wave2 uint16 bit sample, wave3 uint16 bit sample, wave1 uint16 bit sample, wave2 uint16 bit sample, wave3 uint16 bit sample .....etc
So I would like to use this new file which should hopefully speed things up, but I know I will need to implement an interrupt somewhere, I am confused/fearful that setting an interrupt that outputs the data to the DAC at a 6.4kHz will damage the sd card, if the interrupt is called while a buffer is being updated from the sdcard.
This is a really long post sorry. What are your thoughts Mike?
What are your thoughts Mike?
Well I think you are going about it all wrong.
-
You do not need 224000 rows of data, to get 35 seconds of waveform. You need a look up table of only 256 entries and play the same samples over and over. To get the three phase signal all you need is to have three pointers to your look up table. You can generate the look up table in the setup function in SRAM, no need for an SD card.
-
You haven't got HEX data in your SD card you have an ASCII representation of HEX data. This means you have to spend a lot of time making an actual number from the text and opposed to just reading the raw bytes. With raw bytes the data you read is the data you use, so their is no time wasted. However for a digitised sound you will need to read the samples from the SD card. You should put the samples on the SD card by preparing them using the free application Audacity. Save them as WAV files and when you are reading the SD card miss the 40 or so bytes of header.
However, if you want three interleaved sound samples then you will have to write a simple program in Processing to interleave the samples from the three sounds. And then save the results in a file and then transfer that to the SD card. -
"My program has quite a few lines so I will try and make this a bit easier and only post the code concerning the sd card playback." While I appreciate the intent this actually makes things harder because I can't see what is driving the sampling rate.
am confused/fearful that setting an interrupt that outputs the data to the DAC at a 6.4kHz will damage the sd card, if the interrupt is called while a buffer is being updated from the sdcard.
Access is done to the SD card while the interrupts are disabled, but even if they were not then you can't damage the SD card if your circuit is correct. We have not seen this yet.
but I know I will need to implement an interrupt somewhere,
not necessarily with the files in a proper format you will have time to just get a sample, output it and then have a small padding delay to control the output sample rate. Otherwise the interrupt service routine could do both the read and write one after the other, with the timer generating the sample rate.
Thank you for the feedback Mike.
- The other code that I have ommited is actually a manual mode where I have saved 360 entries that represent a sine wave in an array, the user is then able to output 8 sine waves at a time, and I use a variable delay that controls the frequency of the output. The 35 second data that I have on the sd card has a lot more variance. For example, the waveforms start at a frequency of 50Hz, then they slow down at specific intervals to 45 Hz, then up to 55Hz at specific intervals, then the amplitude of each waveform steps up etc.. The 35 second data is actually a benchmark test used for phasor measurement units used in the power industry. So my project has 2 modes, a manual mode were the user can choose 8 sine waves and output them with different phase characterisitics. The second mode is the playback mode, where I want to be able to play back this benchmark test from the sd card, I also want to be able to play back other system events such as short circuit faults and frequency events. The sample frequency of these events are all 6.4kHz, all of the variance in frequency and amplitude all all contained in the samples, so as long as I am able to output to the DAC at 6.4kHz then that is perfect.
2.Yes this is absolutely true, and completely wasteful, I now have compiled a .bin file that is just filled with the 16 bit samples. This has decreased the file size down to 1.3MB instead of the 3.5MB ascii hex CSV file.
- Honestly there isnt much to look at in the other code, it literally is just lots of data saved for the manual mode sinewave entries. Sample frequency is rudimental, all I have done is put in a delayMicroseconds in the loop. The user is able to change this value and output at different frequencys. As I said, very rudimental. Perhaps if I get to know interupts I can go back and update my code to use an interrupt here as well. If you insist in seeing all of it then please just say and I will post it, but to be honest it is horrible code, and horribly formatted.
4.Hopefully this would be great.
So right at this moment I have my new .bin file and I would like to read this from the card and output it at 6.4kHz, the code I have now is useless. I have read about sdfat raw reads? I dont know how to implement though, would you know much about that Mike?