Hello everyone,
I am working on a project called VITAL-AI, an offline AI medical assistant prototype based on an ESP32, an INMP441 I2S microphone, an OLED SSD1306 display, a MAX98357A I2S amplifier, and a local PC server running Whisper/Ollama/Piper TTS.
The global pipeline is already working:
text
ESP32 records audio
→ sends WAV file to FastAPI server over WiFi
→ Whisper transcribes
→ Ollama generates a response
→ Piper TTS generates response.wav
→ ESP32 downloads and plays response.wav through MAX98357A
The WiFi, server, upload, response download, OLED, and speaker pipeline are working.
The current blocking problem is the INMP441 microphone audio capture.
I am trying to record audio from the INMP441 into a WAV file stored temporarily in SPIFFS, then upload it to the PC server. The server saves the received file as:
text
C:\Users\HP COM\piper\last_input.wav
When I play this file on the PC, I either get:
- complete silence,
- or digital noise / clicks / beeps,
- but never a clear voice.
Whisper therefore returns an empty transcription.
Hardware used
- ESP32 DevKit V1 / ESP-WROOM-32
- INMP441 I2S microphone
- MAX98357A I2S amplifier
- 3W speaker
- OLED SSD1306 128x64 I2C
- Push button
- SPIFFS used instead of SD card for temporary WAV storage
Current INMP441 wiring
I have tested multiple wiring configurations, but the main current one is:
text
INMP441 ESP32
-------------------------
VDD → 3.3V
GND → GND
SCK → GPIO33
WS → GPIO32
SD → GPIO25
L/R → GND
With this configuration I use:
C++
#define MIC_BCLK 33
#define MIC_WS 32
#define MIC_SD 25
I have also tested another commonly shown wiring:
text
INMP441 ESP32
-------------------------
VDD → 3.3V
GND → GND
SCK → GPIO32
WS → GPIO25
SD → GPIO33
L/R → GND
which corresponds to:
C++
#define MIC_BCLK 32
#define MIC_WS 25
#define MIC_SD 33
That configuration gave some activity in RMS logs, but the recorded WAV sounded like digital noise / beeps, not voice.
MAX98357A wiring
The speaker pipeline seems to work, so I am not focusing on that now.
text
MAX98357A ESP32
-------------------------
VIN → 5V
GND → GND
DIN → GPIO26
BCLK → GPIO27
LRC → GPIO14
I may later share clocks with the mic, but for now the amplifier seems to play the response audio correctly.
OLED wiring
text
OLED SSD1306 ESP32
-------------------------
VCC → 3.3V
GND → GND
SDA → GPIO21
SCL → GPIO22
What works
The following parts work:
text
ESP32 boots correctly
OLED works
WiFi works
mDNS works
FastAPI server receives the uploaded WAV
Server returns HTTP 200 OK
Piper generates response.wav
ESP32 downloads response.wav
MAX98357A plays the response audio
The server logs show:
text
[AUDIO] Received file: ~150 KB to ~400 KB
[DEBUG] Saved: C:\Users\HP COM\piper\last_input.wav
[STT] ""
So the server is receiving a valid WAV file size-wise, but the audio content is not usable.
Problem symptoms
Case 1: using this config
C++
#define MIC_BCLK 33
#define MIC_WS 32
#define MIC_SD 25
#define MIC_SHIFT 11
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT
with:
text
L/R → GND
The generated last_input.wav is silent or almost silent.
Serial output shows many values like:
text
[MIC] rms=0 peak=0
[MIC] rms=0 peak=0
[MIC] rms=15 peak=245
[MIC] rms=0 peak=0
No voice is recorded.
Case 2: using other shift/channel combinations
For example:
C++
MIC_SHIFT = 8, 11, 14, 16
I2S_CHANNEL_FMT_ONLY_LEFT or ONLY_RIGHT
I2S_COMM_FORMAT_I2S or I2S_COMM_FORMAT_I2S_MSB
I often get digital-looking values such as:
text
[MIC] chunk=256 rms=2048 peak=32768
[MIC] chunk=256 rms=1024 peak=16384
[MIC] chunk=256 rms=512 peak=8192
[MIC] chunk=256 rms=0 peak=1
The resulting WAV sounds like:
text
grinding / clicking / beeping / digital noise
but not voice.
Current I2S setup function
This is the current microphone setup I am testing:
C++
void setupMic() {
uninstallI2S();
i2s_config_t cfg = {
.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
.sample_rate = SAMPLE_RATE,
.bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT,
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
.communication_format = I2S_COMM_FORMAT_I2S,
.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
.dma_buf_count = 8,
.dma_buf_len = 256,
.use_apll = false,
.tx_desc_auto_clear = false,
.fixed_mclk = 0
};
i2s_pin_config_t pins = {
.bck_io_num = MIC_BCLK,
.ws_io_num = MIC_WS,
.data_out_num = I2S_PIN_NO_CHANGE,
.data_in_num = MIC_SD
};
i2s_driver_install(I2S_NUM_0, &cfg, 0, NULL);
i2s_set_pin(I2S_NUM_0, &pins);
i2s_zero_dma_buffer(I2S_NUM_0);
micInstalled = true;
speakerInstalled = false;
Serial.printf(
"[MIC] setup done: BCLK=%d WS=%d SD=%d SR=%d CH=LEFT\n",
MIC_BCLK,
MIC_WS,
MIC_SD,
SAMPLE_RATE
);
}
I2S uninstall function
Because I alternate microphone and speaker on the same I2S peripheral:
C++
void uninstallI2S() {
if (micInstalled || speakerInstalled) {
i2s_driver_uninstall(I2S_NUM_0);
micInstalled = false;
speakerInstalled = false;
delay(80);
}
}
Current audio constants
C++
#define SAMPLE_RATE 16000
#define MAX_RECORD_SECONDS 15
#define SILENCE_THRESHOLD 180
#define SILENCE_TIMEOUT_MS 1500
#define MIN_RECORD_SAMPLES 8000
#define MIC_SHIFT 11
I have tried several values for MIC_SHIFT:
text
8, 11, 13, 14, 16
but I still do not get clean voice.
Current recording function
I record to SPIFFS as a WAV file. I first write an empty 44-byte WAV header, then write PCM samples, then seek back to update the WAV header.
C++
bool recordAudioToSPIFFS() {
if (!micInstalled) setupMic();
if (SPIFFS.exists("/voice.wav")) {
SPIFFS.remove("/voice.wav");
}
File f = SPIFFS.open("/voice.wav", FILE_WRITE);
if (!f) {
Serial.println("[REC] Cannot create /voice.wav");
return false;
}
uint8_t emptyHeader[44] = {0};
f.write(emptyHeader, 44);
setEmotion(ECOUTE);
digitalWrite(LED_BLUE, HIGH);
digitalWrite(LED_RED, HIGH);
Serial.println("[REC] Hold button and speak...");
int32_t i2sBuffer[256];
size_t bytesRead = 0;
uint32_t totalSamples = 0;
unsigned long startMs = millis();
unsigned long lastSoundMs = millis();
bool wasSpeaking = false;
while (true) {
if (millis() - startMs >= MAX_RECORD_SECONDS * 1000UL) {
Serial.println("[REC] Max duration reached");
break;
}
esp_err_t res = i2s_read(
I2S_NUM_0,
i2sBuffer,
sizeof(i2sBuffer),
&bytesRead,
portMAX_DELAY
);
if (res != ESP_OK || bytesRead == 0) {
Serial.println("[REC] I2S read error");
continue;
}
int samples = bytesRead / 4;
double sumSq = 0;
int peak = 0;
for (int i = 0; i < samples; i++) {
int32_t raw = i2sBuffer[i];
int16_t sample = (int16_t)(raw >> MIC_SHIFT);
f.write((uint8_t*)&sample, 2);
int absSample = abs(sample);
if (absSample > peak) peak = absSample;
sumSq += (double)sample * (double)sample;
totalSamples++;
}
int rms = (int)sqrt(sumSq / samples);
int meter = constrain(rms / 200, 0, 20);
String bar = "";
for (int i = 0; i < meter; i++) bar += '=';
Serial.printf(
"[MIC] chunk=%d rms=%d peak=%d [VAD] %s\n",
samples,
rms,
peak,
bar.c_str()
);
if (rms > SILENCE_THRESHOLD) {
wasSpeaking = true;
lastSoundMs = millis();
}
if (wasSpeaking && digitalRead(BUTTON_PIN) == HIGH) {
Serial.println("[REC] Button released");
break;
}
if (wasSpeaking && millis() - lastSoundMs > SILENCE_TIMEOUT_MS) {
Serial.println("[REC] Silence detected");
break;
}
}
digitalWrite(LED_BLUE, LOW);
digitalWrite(LED_RED, LOW);
uint32_t dataSize = totalSamples * 2;
writeWavHeader(f, dataSize);
f.close();
float duration = (float)totalSamples / SAMPLE_RATE;
Serial.printf(
"[REC] Done: %lu samples, %.1fs, %lu bytes\n",
totalSamples,
duration,
dataSize + 44
);
return totalSamples > MIN_RECORD_SAMPLES;
}
WAV header function
C++
void writeWavHeader(File &f, uint32_t dataSize) {
uint32_t fileSize = dataSize + 36;
uint32_t byteRate = SAMPLE_RATE * 2;
f.seek(0);
f.write((const uint8_t*)"RIFF", 4);
f.write((uint8_t*)&fileSize, 4);
f.write((const uint8_t*)"WAVE", 4);
f.write((const uint8_t*)"fmt ", 4);
uint32_t fmtSize = 16;
uint16_t audioFormat = 1;
uint16_t numChannels = 1;
uint32_t sampleRate = SAMPLE_RATE;
uint16_t blockAlign = 2;
uint16_t bitsPerSample = 16;
f.write((uint8_t*)&fmtSize, 4);
f.write((uint8_t*)&audioFormat, 2);
f.write((uint8_t*)&numChannels, 2);
f.write((uint8_t*)&sampleRate, 4);
f.write((uint8_t*)&byteRate, 4);
f.write((uint8_t*)&blockAlign, 2);
f.write((uint8_t*)&bitsPerSample, 2);
f.write((const uint8_t*)"data", 4);
f.write((uint8_t*)&dataSize, 4);
}
Example logs
When I get digital noise, the serial monitor shows values like:
text
[MIC] chunk=256 rms=2048 peak=32768 [VAD] ==========
[MIC] chunk=256 rms=1024 peak=16384 [VAD] =====
[MIC] chunk=256 rms=512 peak=8192 [VAD] ==
[MIC] chunk=256 rms=0 peak=1 [VAD]
When I get silence, I see:
text
[MIC] chunk=256 rms=0 peak=0 [VAD]
[MIC] chunk=256 rms=0 peak=0 [VAD]
[MIC] chunk=256 rms=15 peak=245 [VAD]
In both cases the server receives the file:
text
[AUDIO] Received file: 100 KB to 400 KB
[DEBUG] Saved: C:\Users\HP COM\piper\last_input.wav
But the file is either silence or digital noise.
My questions
- What is the correct I2S configuration for INMP441 on ESP32 using Arduino core?
- Should I use:
C++
I2S_COMM_FORMAT_I2S
or
C++
I2S_COMM_FORMAT_I2S_MSB
- Should INMP441 data be extracted with:
C++
raw >> 8
raw >> 11
raw >> 14
raw >> 16
or another method?
4. Is my WAV writing method correct?
5. Is SPIFFS too slow for writing audio this way?
6. Should I write the PCM data in blocks instead of one sample at a time?
7. Is it better to test the INMP441 alone first, without the speaker and without SPIFFS?
8. Could the issue be caused by alternating microphone and speaker on the same I2S peripheral?
What I would appreciate
I would really appreciate a minimal working example for:
text
ESP32 + INMP441
16 kHz
mono
WAV output
record 5 seconds
save to SPIFFS or send to Serial/WiFi
Or at least a known-good configuration for:
C++
i2s_config_t
i2s_pin_config_t
sample conversion from int32_t I2S to int16_t PCM
Thank you very much for any help. This project is for a university AI-for-health prototype, and I am very close to finishing the complete system. The last blocking issue is getting clean voice audio from the INMP441.