Ultrasonic "Sensory Extension" Wearable

Hello,

I'm prototyping a sensory extension device that would collect ultrasonic frequency information between ~ 15kHz - 45kHz. My intent is to trigger a specific spatiotemporal pattern, on a vibrotactile array located on the back of the hand, in response to a given subdivided frequency range within the overall 15-45kHz range (for example: trigger tactile pattern x, when frequency range 20-23kHz is detected). It's pretty much an tactile-to-auditory sensory augmentation setup.

I'm having a hard time finding a breakout board that's compatible with the ultrasonic MEMS mic that I require (Ultrasonic MEMS Sensor SPU0410LR5H, http://www.digikey.com/product-detail/en/knowles/SPU0410LR5H-QB-7/423-1139-1-ND/2420983). AND since I have no experience with electrical engineering, I'm also having a hard time figuring out how to procure a custom PCB with the appropriate MEMS footprint. Once I have the mic breakout figured out, I believe I can just use this Sparkfun opamp breakout to boost the signal: https://www.sparkfun.com/products/9816.

IF I'm able to work out the above issues, I then need to find the right frequency analysis code. Could I use the Simple Audio Frequency Meter library to do this? I understand that I would have to increase the bandwidth parameters...would this also be possible?

If not, what would be your recommendation for such a sensor application, with regards to either the hardware and/or software?

I hope this is not too confusing:)

Best, Nick

Could I use the Simple Audio Frequency Meter library to do this?

No. I think that will only give you a single frequency measurement based on zero crossing times. You probably need to go to an FFT. However, you will have to up the sample rate to 100KHz to begin to get something meaningful at the top range. Have you worked out how you are going to do that?

believe I can just use this Sparkfun opamp breakout to boost the signal

Yes if you remove those capacitors, it says in the board description:-

The bandwidth is set to 15.9kHz by a pair of feedback capacitors, or over 100kHz with the caps removed.

I was thinking about implementing the Arduino FHT library. If I set the ADC prescale to 16 (http://forum.arduino.cc/index.php?topic=6549.msg51570#msg51570), I can get a sampling rate of ~77kHz. I'll be happy with max detectable frequency of ~38.5kHz.

Does this sound feasible?

And yes, I'll remove C1 and C3 on the Sparkfun opamp.

Yes it sounds feasible but you could consider an external A/D to increase the sample rate. The only problem I see now is that microphone, it is a surface mount device with the contacts under the device. So even if you had a custom PCB then you would need to flow solder it. While this can be done at home it is a tricky technique to get right on a delicate device like this.

The only other thing you could do with it is called the dead bug technique. Which is to mount it upside down and solder fine wires to the contacts.

So, after minimal success trying to implement the Ultrasonic MEMS mic (Knowles SPU0410LR5H) I realized that the functioning Adafruit MEMS mic breakout works just fine for my purposes (surprisingly, it receives sound signals up to about 22kHz).

I’ve also semi-successfully combined/tweaked the code of Open Music Labs FHT library () with Patric Laub’s Haptic Interface Arduino Prototype ().

I’m using a 3x3 array of vibratory motors (image attached). Currently, my program can trigger a specific motor pattern in response to a specific frequency (e.g., 18kHz frequency received → top row of motors turn on simultaneously for duration of stimulus). I can’t figure out how to pulse the patterns (say, every 0.5 seconds or so). Additionally, can’t quite figure out how to, instead of simultaneous motor activation, trigger spatiotemporal motor sweeps (i.e., top-left motor ON then OFF, top-middle motor ON then OFF, top-right motor ON then OFF)

Here’s the code so far:

//Sensory_Extension_Prototype
//By Nick Gonyea
//Code derived from OpenMusicLab’s FHT library & PatricLaub’s Haptic Interface Arduino Prototype
#define LOG_OUT 1 // use the log output function
#define FHT_N 256 // set to 256 point fht
#include <FHT.h> // include the library
//These variables allocate different ports to each vibration motor
int motor1 = 5;
int motor2 = 6;
int motor3 = 13;
int motor4 = 7;
int motor5 = 9;
int motor6 = 12;
int motor7 = 8;
int motor8 = 10;
int motor9 = 11;
void setup() {

  • Serial.begin(115200);*
  • TIMSK0 = 0; // turn off timer0 for lower jitter*
  • ADCSRA = 0xe5; // set the adc to free running mode*
  • ADMUX = 0x40; // use adc0*
  • DIDR0 = 0x01; // turn off the digital input for adc0*

}
void loop() {

  • while(1) { // reduces jitter*
  • cli(); // UDRE interrupt slows this way down on arduino1.0*
  • for (int i = 0 ; i < FHT_N ; i++) { // save 256 samples*
  • while(!(ADCSRA & 0x10)); // wait for adc to be ready*
  • ADCSRA = 0xf4; // restart adc w/ prescale set to 16 (sampling rate: ~77kHz)*
  • byte m = ADCL; // fetch adc data*
  • byte j = ADCH;*
  • int k = (j << 8 ) | m; // form into an int*
  • k -= 0x0200; // form into a signed int*
  • k <<= 6; // form into a 16b signed int*
    fht_input = k; // put real data into bins
    * }*

* fht_window(); // window the data for better frequency response*
* fht_reorder(); // reorder the data before doing the fht*
* fht_run(); // process the data in the fht*
* fht_mag_log(); // take the output of the fht*
* sei();*
* //Serial.write(255); // send a start byte*
* //Serial.write(fht_log_out, FHT_N/2); // send out the data*

* if (fht_log_out[60] > 130){
_
Serial.println(“18 kHz”);_
_
displaySetting1(motor1, motor2, motor3, motor4, motor5, motor6, motor7, motor8, motor9);*_

* }*
* else if (fht_log_out[63] > 130){
_
Serial.println(“19 kHz”);_
_
displaySetting2(motor1, motor2, motor3, motor4, motor5, motor6, motor7, motor8, motor9);*_

* }*
* else if (fht_log_out[67] > 130){
_
Serial.println(“20 kHz”);_
_
displaySetting3(motor1, motor2, motor3, motor4, motor5, motor6, motor7, motor8, motor9);*_

* }*
* else if (fht_log_out[70] > 115){
_
Serial.println(“21 kHz”);_
_
displaySetting4(motor1, motor2, motor3, motor4, motor5, motor6, motor7, motor8, motor9);*_

* }*
* else{*
* Serial.println(“No detectable frequency”);*
* displaySetting0(motor1, motor2, motor3, motor4, motor5, motor6, motor7, motor8, motor9);*

* }*

* }*
}
int displaySetting1(int mtr1, int mtr2, int mtr3, int mtr4, int mtr5, int mtr6, int mtr7, int mtr8, int mtr9){
_ //Serial.println(“Display setting 1”);_
* analogWrite(mtr1, 0);*
* analogWrite(mtr2, 0);*
* analogWrite(mtr3, 0);*
* analogWrite(mtr4, 255);*
* analogWrite(mtr5, 255);*
* analogWrite(mtr6, 255);*
* analogWrite(mtr7, 0);*
* analogWrite(mtr8, 0);*
* analogWrite(mtr9, 0);*

}
int displaySetting2(int mtr1, int mtr2, int mtr3, int mtr4, int mtr5, int mtr6, int mtr7, int mtr8, int mtr9){
_ //Serial.println(“Display setting 2”);_
* analogWrite(mtr1, 0);*
* analogWrite(mtr2, 0);*
* analogWrite(mtr3, 255);*
* analogWrite(mtr4, 0);*
* analogWrite(mtr5, 255);*
* analogWrite(mtr6, 0);*
* analogWrite(mtr7, 255);*
* analogWrite(mtr8, 0);*
* analogWrite(mtr9, 0);*

}
int displaySetting3(int mtr1, int mtr2, int mtr3, int mtr4, int mtr5, int mtr6, int mtr7, int mtr8, int mtr9){
_ //Serial.println(“Display setting 3”);_
* analogWrite(mtr1, 0);*
* analogWrite(mtr2, 255);*
* analogWrite(mtr3, 0);*
* analogWrite(mtr4, 0);*
* analogWrite(mtr5, 255);*
* analogWrite(mtr6, 0);*
* analogWrite(mtr7, 0);*
* analogWrite(mtr8, 255);*
* analogWrite(mtr9, 0);*

}
int displaySetting4(int mtr1, int mtr2, int mtr3, int mtr4, int mtr5, int mtr6, int mtr7, int mtr8, int mtr9){
_ //Serial.println(“Display setting 4”);_
* analogWrite(mtr1, 255);*
* analogWrite(mtr2, 0);*
* analogWrite(mtr3, 0);*
* analogWrite(mtr4, 0);*
* analogWrite(mtr5, 255);*
* analogWrite(mtr6, 0);*
* analogWrite(mtr7, 0);*
* analogWrite(mtr8, 0);*
* analogWrite(mtr9, 255);*

}
int displaySetting0(int mtr1, int mtr2, int mtr3, int mtr4, int mtr5, int mtr6, int mtr7, int mtr8, int mtr9){
_ //Serial.println(“Display setting 0”);_
* //Inactivate all motors*

* analogWrite(mtr1, 0);*
* analogWrite(mtr2, 0);*
* analogWrite(mtr3, 0);*
* analogWrite(mtr4, 0);*
* analogWrite(mtr5, 0);*
* analogWrite(mtr6, 0);*
* analogWrite(mtr7, 0);*
* analogWrite(mtr8, 0);*
* analogWrite(mtr9, 0);*

}
Any help would be very much appreciated! Also, besides my specific code help questions, if anyone sees ways in which to optimize the current code, please let me know:)
Thanks,
Nick
3x3array.jpg