High Quality, Ultra-Low Latency, with I2S/TDM and USB

[aarond-gh]

There's the Teensy Audio Library, which is made for PJRC's Teensy 3.x and 4.x, and uses Node-RED for a graphical editor, and works just fine for just playing around, making synthesizers, and other things that don't require the dynamic range of a pro live concert rig. But I have some projects that do need more than what it's hard-coded for:

• 16-bit integer
• 44.1kHz
• Stereo on USB
• 128-sample buffer
  • 2.9ms latency at 44.1kHz, plus the converters' group delay

Is there a different library for the Teensy 4.1 (or the NXP i.MX RT1062 chip that's on it) that offers:

• 32-bit float or better processing
• User-selected bit depth on I2S/TDM and USB separately, including at least 16, 24, and 32 bits
• User-selected sample rate, including at least 48kHz and 96kHz
• User-selected channel count on I2S/TDM and USB separately, including at least 2 to 8
• No buffer (or 1-sample buffer, depending on how you think of it), which results in a 3-sample pipeline on I2S/TDM: clock in, process, clock out
  • 31.25us (0.03125ms) latency at 96kHz, plus the converters' group delay

If not, how hard is it to make one?

--

I know that I2S and TDM are different protocols, but they're so similar that I imagine that the only difference to the user code would be the channel count.

For example, I made a single driver on the Raspberry Pi Pico's "Programmable I/O" module that does do both, based on its options for bit depth, channel count, and frame-sync polarity. Otherwise, same signals and timing, provided that it's okay for the TDM frame sync to last for an entire channel.

--

I imagine that the sketch would end up something like this, possibly using my own DSP filter functions:

void setup()
{
    // put your setup code here, to run once:

    Serial.begin(115200);   // USB Serial

    Audio.begin(...);       // I2S/TDM and USB Audio
}


void loop()
{
    // put your main code here, to run repeatedly:
    // lowest priority, preempted by USB and DSP interrupts

    Serial.println("debug messages");

    // also calculate DSP coefficients and then:
    newCoefficientsAvailable = true;
    // to pass to DSP code

    // and whatever else this project needs, like a front panel with:
    //   - pots
    //   - buttons
    //   - quadrature encoders
    //   - LED's
    //   - motorized faders (linear pots)
    //   - etc.
}


void dsp()
{
    // put your dsp code here, to run once for each sample:

    // probably a top-priority interrupt from I2S/TDM new frame
    // preempts loop(), USB, and everything else, user-visible or not

    LED_off();  // DSP load and sample rate indicator

    if (newCoefficientsAvailable)
    {
        // grab new coefficients

        newCoefficientsAvailable = false;
    }

    // magic numbers are channel selections
    // audio_t is typedef'ed as a 32-bit float or better
    audio_t sample = fromADC(3) * gain_coeff;
    sample = filter1(sample, filter1_state, filter1_coeff);
    toUSB(sample, 5);

    sample += fromUSB(2);
    toDAC(  filter4(filter3(filter2(sample,
                filter2_state, filter2_coeff),
                filter3_state, filter3_coeff),
                filter4_state, filter4_coeff),
            1);

    LED_on();   // DSP load and sample rate indicator
}