Thank you fat16lib. I will try ending the bluetooth connection before calling analogBinLogger, and report back. However, I am left confused how to get the data to the phone. I don't think I can reconnect 32000x times to send BLE packets (20bytes long for 20kB total data).
Do you know if SdFat is compatible with wire.h? I have seen a few posts regarding this compatibility. If the different SPI devices are located on different arduinos, will I still see conflict?
Dangit. There is no end() function the Adafruit_BLE_UART library. I hope the dual arduinos approach can work...
SD cards are SPI devices and can't use I2C. I2C is also way too slow, Wire runs at 100 kHz. SPI runs at 8 MHz, 80 times faster.
My new software SPI runs at about 2 MHz with SdFat.
I tried AnalogBinLogger with software SPI. I logged 4 pins at 5 kHz on a Mega using an Adafruit shield that has no ISP connector, just connections to pins 10-13. Hardware SPI on Mega is the ISP or pins 50-53.
It ran successfully with no overruns.
Sample pins: 0 1 2 4
ADC bits: 10
ADC clock kHz: 500
Sample Rate: 5000.00
Sample interval usec: 200.0000
Creating new file
Erasing all data
Logging - type any character to stop
Truncating file
File renamed: analog02.bin
Max block write usec: 5196
Record time sec: 56.500
Sample count: 282481
Samples/sec: 4999.66
Overruns: 0
Done
A Bluetooth library could still cause problems if interrupts are disabled too long.
You can enable software SPI in SdFat by editing SdFatConfig.h and set this:
#define SD_SPI_CONFIGURATION 2
It doesn't seem the Bluetooth library will work. I could not get it working reliably with <SD.h>. I don't understand enough of the programming in SdFat to modify to work with the Adafruit BLE library.
I didn't mean use I2C for SD connections. I meant connect the SPI devices on separate Arduinos and send the data between the arduinos via I2C, so there is not SPI conflict. I am hoping I2C does not have conflict with SPI.
In this configuration, master Arduino has the BLE breakout, and slave Arduino has the SD + analogBinLogger. Communication via I2C to send data between master and slave. Master sends data to iOS app.
After logData() has been called, I would use a modified dumpData() function, to transmit the recorded data so master via i2c, instead of printing to serial. I'm OK with I2C slowing down dumpData() loop, provided that this will work.
Madness? I think so. Perhaps understanding how to use <SD.h> and writing analog signals using adc() is more reasonable? However this is my problem that I need solved, it seems like using a Due might work, so that I can have enough RAM for 2x 5kHz measurement @ 1 second. But then getting the BLE to play with the Due is another story...
Why can't you just use a simple Bluetooth modem like this?
These Bluetooth wireless modems work as a TTL serial (RX/TX) pipe, as a replacement for serial cable connections.
You just hook it to the serial pins on Arduino
I know modems like BlueSMiRF have been used with ios and Andriod. You need to make sure you find one that works with ios.
Looks like this device works with ios and connects to Arduino serial.
The BearLab shield appears to connect to SPI.
Bluetooth classic requires additional enrollment to Apple MFi program, or phone must be jailbroken. These requirements will not work for this application.
I added MasterWriter/SlaveReceiver example to analogBinReader() and it was unsuccessful. Using Wire.onReceive(receiveEvent), A serial.print() statement works within receiveEvent() but if logData() is placed inside, logData() does not work from Serial port or from Wire.h call. You were right, I was not able to use I2C to call the analogBinLogger() library.
I just learned ICSP wires digital pins 11-13 to the hardware SPI. Explains why SdFat was working without declaring Software SPI.
On the Mega, if I ran software SPI for the shield (bypassing ICSP headers) and BLE shield on 50-53 will the SPI calls still conflict? I am going to try this next approach. post #5 suggests having run on software SPI might help these two libraries work (SD Library conflicts with other SPI devices? - Networking, Protocols, and Devices - Arduino Forum)
This device is BLE, not Bluetooth classic. See the ios section of the link. This device connects to the Serial pins.
You can now incorporate Bluetooth 4.0 Low Energy (BLE) technology easily in your project with our BLE Mini. BLE Mini requires only a serial port for communication so it supports all major development platforms that have UART interface including Arduino, Raspberry Pi, Netduino, BeagleBone etc.
On the Mega, if I ran software SPI for the shield (bypassing ICSP headers) and BLE shield on 50-53 will the SPI calls still conflict?
I often use an Adafruit data logging shield on a Mega with software SPI it connects to pins 10-13 and does not access the Mega hardware SPI on pins 50-53 which are also connected to the ISP pins. The Adafruit data logging shield has no ISP connector. There is no conflict with hardware SPI pins.
There are several problems with the Adafruit library so it may conflict with any approach to fast logging. The Adafruit Bluetooth shield uses an interrupt and is slow so it may block reading the adc fast enough unless you can disable Adafruit Bluetooth while logging.
The reason you can't share the SPI bus with the Adafruit Bluetooth shield is that it does not set SPI options properly. The Adafruit Bluetooth library needs to be updated to use the new Arduino 1.6x SPI library with SPI transactions.
Be aware that logging two pins at 5 kHz can't even be done with the standard analogRead() on a Mega. I increase the ADC clock by a factor of up to four. Any extra latency will prevent logging at the speed you want.
If you are going to play with sending fast data via any serial interface (uart, BLE, SPI, I2C) always start with calculating the bandwidth (kBytes/sec) you need. Then you may simply simulate the data stream to assess, whether your link is even capable to transfer such amount of data reliably, or not.
An example - I want to stream 5000 samples per second, each sample consists of 4 ADC readings.
You may pack in four 10bit ADC values into 5 bytes when lucky, and you will not use any samples separator like new line or carriage return.
So the raw speed you need to achieve with this example is 5000 x 5 = 25kBytes per second.
Now you may try to send 25kBytes/second via your link, ie BLE or UART or whatever else, in a simple loop, to verify whether it works reliably.
The BLE mini you have suggested looks pretty good. It seems like the best solution to integrate BLE with the SdFat library! Also leaves me room to add more potentially conflicting SPI devices, heh...
However, it sounds like the Adafruit BLE module is salvageable if I can connect the SD and SdFat to software SPI, and the BLE to hardware SPI, then the two will not be conflicting... is this true? Actually BLE to software SPI, and BLE to hardware SPI seems more optimized.
I am really impressed by your work, which is why I would like to use this library.
Thanks all for your responses, this is all very helpful to me!
I was just trying to understand the code of the datalogger (AnalogBinLogger)
just my question is about the logic that combines the timer and the "ADC done interrupt" , as I understand in ISR(ADC_vect) is performing a new conversion when the last conversion is done, so some how this will keep running according to the adc speed which can be determined by the prescaler and the CPU frequency, then the timer some how is used to get the the samples according to the sampling intervals. how is the timer " connected" to ISR(ADC_vect) .
fat16lib could you just explain how it works, the general concept without coding, the flow of the timer and the adc interrupt ?
and just to confirm: I guess timer 0 is just for Millis and Micros and timer 1 is the one that related to the sampling frequency and ish ?
fat16lib could you just explain how it works, the general concept without coding, the flow of the timer and the adc interrupt ?
The ADC conversion for the first pin is started by timer 1, not by an interrupt. This allows the jitter in the first conversion to be less than one CPU cycle no matter what else is happening. The ADC conversions for the remaining pins are started in the ADC done ISR.
See the AVR datasheet to see how timer 1 can trigger an ADC conversion.
thanks fat16lib that helped a lot,
I am trying to use some of these techniques to send data through an Ethernet shield to a web page in fixed intervals
another question fat16lib, what is the flow of writing these blocks to the memory card, I mean like, lets say for the Mega we have 13 blocks that use the RAM space, my question is: when do you write these blocks to the memory? after they all get full? and if so, while writing into the SD card, if these blocks are full, where do you store the new samples?
that would help and save me a great portion of time. I need just the flow and the idea no coding
thanks
I write a block to the SD as soon as possible after it is full.
Hi Fat16lib,
I was wondering if its possible to use this to find data from an mpu 6050 connected to an Uno and Adafruit data logger at a high sample rate (about 1000Hz)?
Also, whenever i try to run the analogbinlogger example, it can never locate any of the files such as SDFat.
Thanks.
I was wondering if its possible to use this to find data from an mpu 6050 connected to an Uno and Adafruit data logger at a high sample rate (about 1000Hz)?
No, this logger can only log analog pins. The mpu 6050 is an I2C device.
Also, whenever i try to run the analogbinlogger example, it can never locate any of the files such as SDFat.
The SdFat folder must be in the wrong location. On Windows the folder need to be here
C:\Users<user name>\Documents\Arduino\libraries\SdFat
Damn.
I thought as much but worth an ask. Thanks for the speedy reply. Seems like its impossible to log the data onto an SD at 1000Hz
Hi @fat16lib
first of all I would like to thank you because you did a great thing to humanity.
I just wanna know how to use the: bintocsv program, I mean if I want to do the conversion using the PC not the Arduino
when I open up the bintocsv.exe, it remains for less than one second and then it disappears
when I open up the bintocsv.exe, it remains for less than one second and then it disappears
See readme.txt
The bintocsv folder contains a PC program for converting binary files to
CSV files. I have included a executable for Windows. Linux and Mac users
can build from the included source files. bintocvs is a command line program.bintocsv binFile csvFile
Here is sample output:
C:\tmp>bintocsv analog01.bin analog01.csv
pinCount: 5
Sample pins: 0 1 2 3 4
ADC clock rate: 500 kHz
Sample rate: 5000 per sec
Sample interval: 200.0000 usec
124620 ADC values read
Here is the start of analog01.csv
Interval,200.0000,usec
pin0,pin1,pin2,pin3,pin4
285,318,308,337,1023
713,570,470,422,1023
900,774,655,562,1023
975,897,801,700,1023
1004,960,896,809,1023
1012,989,949,884,1023
Hiii @fat16lib, in ADC clock frequency Versus ENOB, lets say in the case of 250 KHz ADC clock frequency, the ENOB is 9.5,
when I am converting back to volts again, should I divide by 2^ (10 ) or 2^(9.5) ?
thanks
Regarding the SD card writing, is this an optimized implementation of elm-chan FatFs or is it completely written from scratch for the adc logging?
What currently prevents it from recording data faster than 40 KSps for 1 channel?
For example, I've been thinking of digitizing a camera feed with a ultra fast adc, a UHS class sd card (30MB/s), and a 10Mhz SPI clock to the SD.. what would the expected data rate that could be reliably saved to the card without dropping samples?