I have made a project were I can create 8 sinusoidal waveforms and output them through an 8 channel DAC. I am using a look up table method for the sine waves (which are declared before setup and loop) and I send the values to the DAC using SPI transfers. I would like to take this project further, I have a csv file containing ascii numbers which represent the points on a waveform, much like the sine look up table I have created previously. I would like to save this csv on a micro sd card, and hook it up to my Arduino Due. The csv file has 4 columns of data, each column represents a waveform. Like so-
It seems doable to me. Why though do you store the values rather than calculating them? Don't you calculate the values in the first place to put them in the file?
The data is given to me by someone else and is not calculated by me. They have created the csv file for me. Each of the 4 waves has a sample rate of 6.4kHz. Each value is an ascii representation of a 16 bit number eg all entries are between 0 and 65535. I would like to output this data at 6.4kHz again, to provide a real time output. I havent used an SD card before with arduino, I am unsure of the best way to read the data and output it.
I would be very keen to replace this with a binary file, but I have no idea how to convert a csv file to binary. I am using SPI to send my data to the DAC, if I had a a binary file then I would save some time by not having to convert the ascii values into an unsigned int which I am currently using in my code. Any thoughts on this?
Use a simple python script to convert it to a binary file:
#!/usr/bin/env python3
import csv
with open('in.csv', newline='') as csvfile, open('out.bin','ab') as binaryfile:
csvreader = csv.reader(csvfile, delimiter='\t')
binaryfile.seek(0)
binaryfile.truncate()
for row in csvreader:
for value in row:
binary = int(value).to_bytes(2, byteorder='little')
binaryfile.write(binary)
The Uno has a clock speed of 16MHz. An instruction takes between 1-4 cycles, so the Uno can process roughly 4M instructions per second.
You have four channels of data each at 7kHz, so that is a total of 28kHz. That means you have roughly 4,000,000/28,000 = 143 instructions available to output each reading.
I would be inclined to read the ASCII data and output it to the DAC as fast as you can measuring the time taken to rip through the whole file. You may find you output at more than 7kHZ. In that case it is simpler to stick with using ASCII as you avoid conversion and there are real advantages in being able to look at the raw data.
Could you give me some tips on the best way to read the file? Should I read each line at a time from the card, parse it and convert each value to unsigned int, then send to DAC?
I would try to keep things as simple as possible at first. For me that would mean reading and parsing the file a line at a time. You should be able to time how long it takes to read, parse and output a line. That will let you know if you are close to your target or missing it by miles.
If I found I could not get the data through fast enough then at that point I would consider moving from an ASCII file to a binary file.
If I was only just missing the target then I might explore exactly how reading the SD card is handled e.g. what delays are there, what buffering is being done. Tuning things might let me stay with ASCII.
Just threw this together on an UNO to see if you can get anywhere near 6.4KHz updates.
I am reading values from an array...
So I am thinking that whilst the main() is going on...it could be downloading more off the SD card in to a buffer array...an interrupt triggers the SPI every 156uS (6.4KHz).
PS: This could not use a buffer and just try read the next row from the CSV in time before the next interrupt and store the values in variables and only update and grab the next row once the interrupt has ended...
PSUEDO CODE! Will need checking. Did this quickly...
#include <SPI.h>
unsigned int pos = 0;
boolean updated = false;
unsigned int data[800];
// Make an array of random values...
void setup() {
for (int i = 0; i < 800; i++) {
data[i] = random(0, 35000);
}
// put your setup code here, to run once:
// initialize SPI:
SPI.begin();
TCCR0A = 0;
TCCR0B = 0;
// 16MHz / 6.4KHz = 2500 ticks of the clock...
// Divide clock by 64...= 39 ticks of the clock = 6.4KHz.
// use the /64 pre-scalar...
TCCR0A |= (1 << WGM01) ;
TCCR0B |= (1 << CS01) | (1 << CS00); // divide by 64 clock scalar. CTC mode (top = OCR0A)
OCR0A = 39; // Top value where counter resets to 0 and repeats...
TIMSK0 = 0;
TIMSK0 |= (1 << OCIE0A); // Enable Interrupt mask on Compare A match.
// Enable interrupts...
SREG |= B1000000;
}
void loop() {
/*
* A function that keeps filling the buffer from the SD card...
* i.e. when buffer [0-11] has been sent via SPI...re-fill it from
* the SD card.
* Repeat for buffer[12-23]...etc...etc...
*
*
*/
}
ISR(TIMER0_COMPA_vect) {
pos += 8;
if (pos >= 800) {
pos = 0;
}
for (byte x = 0; x < 4; x++) {
// 8 bit address
SPI.transfer(0xFF);
// 2 byte data.
SPI.transfer(data[pos]+(x*2));
SPI.transfer(data[pos+(x*2) + 1]);
}
}
Timer1 could be used for more accuracy...16 bit resolution with no prescale.
Don't worry about buffering unless you have a problem. If you do have a problem first look and see what buffering is already being done in libraries before perhaps reinventing the wheel.
Interrupts have their place but they are not magic, they 'interrupt' other processes they don't really let you do several things simultaneously. In your case you are dedicated to the single task of pumping data continuously through at high speed so a tight loop that you know completes each iteration in the required time is a better strategy.
ardly:
Interrupts have their place but they are not magic, they 'interrupt' other processes they don't really let you do several things simultaneously. In your case you are dedicated to the single task of pumping data continuously through at high speed so a tight loop that you know completes each iteration in the required time is a better strategy.
They want 6.4KHz. I assume they want it pretty much bang on 6.4KHz (or as close as possilbe).
I see interrupts as the only real way to ensure accuracy.
If they left this to a task in the main(), then it could fire at all different sorts of times.
With the interrupt, you at least know that:
ISR(TIMER0_COMPA_vect) {
pos += 8;
if (pos >= 800) {
pos = 0;
}
for (byte x = 0; x < 4; x++) {
// 8 bit address
SPI.transfer(0xFF);
// 2 byte data.
SPI.transfer(data[pos]+(x*2));
SPI.transfer(data[pos+(x*2) + 1]);
}
}
Is going to happen on every compare match which occurs at the exact same interval (+/- some random and systematic error that is probably fairly low if the system is isolated and kept at a similar temperature).
The buffering is worth thinking about in future maybe. If there was an error on the SD card...it could mean you need to re-fetch a row.
The buffer could still be being processed by the interrupt whilst main() re-attempts to download any troublesome rows...
But again, this may not be needed as suggested by ardly...but could be worth thinking about in future if you need any "post-processing" before outputting.
Johnny010:
They want 6.4KHz. I assume they want it pretty much bang on 6.4KHz (or as close as possilbe).
I see interrupts as the only real way to ensure accuracy.
If they left this to a task in the main(), then it could fire at all different sorts of times.
I disagree If the code cannot process a line of data and output it at 6.4kHz or faster then all is lost and interrupts will not help. If the code can output data at faster than 6.4kHz then all that is required is to delay so that the lower rate is achieved. Avoiding interrupts make the code easier to debug and also avoids interrupt handling which requires processor time and so slows throughput.
Often CSV files have lines of varying length in which case the time to process a line will vary. It might be desirable to pad the columns in the line with spaces so that the number align and each line has exactly the same number of characters.
Using interrupts is definitely the preferred way, and the only way you're going to get a reliable rate.
You'll have some slight overhead, but that's nothing compared to what you'd have to go through to do the timing in your main loop.
If you're just writing out SPI data in the ISR, it's not that much harder to debug.
And definitely not as hard as having to pad with spaces just to get the timing right.
ardly:
I disagree If the code cannot process a line of data and output it at 6.4kHz or faster then all is lost and interrupts will not help. If the code can output data at faster than 6.4kHz then all that is required is to delay so that the lower rate is achieved. Avoiding interrupts make the code easier to debug and also avoids interrupt handling which requires processor time and so slows throughput.
Often CSV files have lines of varying length in which case the time to process a line will vary. It might be desirable to pad the columns in the line with spaces so that the number align and each line has exactly the same number of characters.
They WANT 6.4KHz. Not "Faster than". Timing is important so note the use of interrupts.
Johnny010:
They WANT 6.4KHz. Not "Faster than". Timing is important so note the use of interrupts.
They want to run at 6.4kHz but are concerned that they will not achieve that throughput.
Using interrupts incurs a processing overhead and so will result in reduced throughput. In this application, dedicated to a single task, interrupts don't offer any real advantage.
They should run the code as fast as they can, no delays, no interrupts. Hopefully they will find they can output at faster than 6.4kHz at which point they can easily introduce timing in their main loop. If they don't run at faster than 6.4kHz then they can look at optimising their code, using buffering or using binary files.
Handling timing in the loop would require the use of the millis() and micros() functions. The millis value is updated using a timer, in an ISR @977 Hz (note: not 1kHz!). The deviation can be as large 0.992 ms.
The micros value is calculated using the timer overflow count from the millis ISR combined with the actual value of the timer. Polling this value in the loop is much less accurate than actually doing timing-critical things in an ISR itself.
Handling timing in the loop would require the use of the millis() and micros() functions. The millis value is updated using a timer, in an ISR @977 Hz (note: not 1kHz!). The deviation can be as large 0.992 ms.
The micros value is calculated using the timer overflow count from the millis ISR combined with the actual value of the timer. Polling this value in the loop is much less accurate than actually doing timing-critical things in an ISR itself.
Let's say you write the code using interrupts and it just fails to achieve the the 6.4kHz throughput, what would you do?
The first thing I would do would be to get rid of the interrupts (as well as thinking about optimisation, buffering etc).
I mentioned making the ACSII file lines the same length in an earlier post. That will even out the process time per line, but the main advantage of having fixed line lengths is that it can greatly simplify the buffering, reading and parsing of lines, to increase throughput.
I would use micros() to determine the time it takes to process a line and then use delayMicroseconds() to throttle back the output to 6.4kHz. If I found the line processing time was consistent I might even be able to rid of the calls to micros().
To check the output a file containing square wave data could be used and a scope put on the DAC. I don't know what the application is but I doubt if the difference in frequency between the tight loop and the interrupt would be significant. The tight loop would though be simpler and easier to debug and more likely to hit the throughput target.
ardly:
Let's say you write the code using interrupts and it just fails to achieve the the 6.4kHz throughput, what would you do?
The first thing I would do would be to get rid of the interrupts (as well as thinking about optimisation, buffering etc).
You are not understanding.
Interrupts are triggered via hardware registers.
They are as dead on as you are going to get!
They are triggered when the clock counter (which is incremented directly by the crystal...maybe after a pre-scalar if you set it) matches the value held in OCRxA, OCRxB or TOP (overflow)...it causes the MCU to jump directly to the part in the code where the instructions are held in the ISR_Routine.
It is like a "drop what you are doing and do this right now" hardware trigger.
Running loop() and just hoping it is "fast enough" is crap.
6.4KHz = 2500 clock ticks at 16MHz.
Say they want a 1% accuracy...
25 clock cycles would be enough to balls this up.
That could simply be about 6 -10 instructions in main() loop...
If main() was busy talking to the SD card and there was an error in the transfer...(I don't know if they use a CRC or whatever) then this means a lot more than 6-10 clock cycles difference between this loop() iteration and the next!
Interrupts are specifically for time critical operations...use them.
And PS:
If there was an issue with timing, Timer1 has 16 bit resolution and with a 16MHz crystal un-scaled...
Each increment is 62.5 nano-seconds.
So all you'd do is change the value in OCR1A...
For example, if they are wanting some Serial output as well from main()...then:
"32612 31111 32123 28172"
requires more bytes than:
"1 9 8 7"
to be sent to UART for hardware processing.
This would incur an offset in time again...
By keeping the timing of the SPI constant and reliably in an Interrupt Service Routine...it will allow debugging to be done in loop() that will not effect the timing at all!
Maybe in loop() the OP could have a nice function that gets Serial commands to change the value in OCR1A for example and make changes to the device's function/output whilst not effecting the critical timing part...
If the code cannot process a line of data and output it at 6.4kHz or faster then all is lost and interrupts will not help. If the code can output data at faster than 6.4kHz then all that is required is to delay so that the lower rate is achieved. Avoiding interrupts make the code easier to debug and also avoids interrupt handling which requires processor time and so slows throughput.