Max Transfer Rate of UDP data with Uno R4 WiFi

Projects Networking, Protocols, and Devices
1

I'm trying to use the Uno R4 WiFi to stream audio data to VLC Media Player over UDP. I have it working fine up to 9KHz with a raw PCM stream but cannot get it any higher. If I just up it to 10KHz, it starts stuttering in VLC and giving debug messages about playback too soon.

I've called Serial1.begin(921600); in setup to run the ESP32 bridge at a faster rate, which I thought was the bottleneck, but it doesn't appear to be.

The exact code I'm running is the following, which requires my AutoAnalogAudio library and a microphone attached to pin A1 of the Uno R4 WiFi.

#include <AutoAnalogAudio.h>
#include "WiFiS3.h"
#include <WiFiUdp.h>
#include "secrets.h"

char ssid[] = SECRET_SSID;
char pass[] = SECRET_PASS;
int status = WL_IDLE_STATUS;
const char* myName   = "Arduino-R4";

const char* targetIP = "10.10.1.174";
const uint16_t targetPort = 5004;

// RTP Configuration - 8-bit PCM at 8kHz
const uint8_t RTP_PAYLOAD_TYPE = 96;   // PCMU (will work for raw PCM too)
const uint16_t SAMPLE_RATE = 9000;
const uint16_t SAMPLES_PER_PACKET = 1024;

uint16_t rtp_sequence = 0;
uint32_t rtp_timestamp = 0;
uint32_t rtp_ssrc = 0x12345678;

AutoAnalog aaAudio;
WiFiUDP udp;

#define bufferSize 512

void setup() {
  Serial1.begin(921600);
  Serial.begin(921600);

  modem.begin(); 
  WiFi.setHostname(myName);

  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    Serial.println("Please upgrade the firmware");
  }

  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    status = WiFi.begin(ssid, pass);
    delay(10000);
  }
  Serial.print("IP Address: ");
  Serial.println(WiFi.localIP());

  Serial.println("Analog Audio Begin");
  pinMode(5, OUTPUT);
  digitalWrite(5, HIGH);
  pinMode(6, OUTPUT);
  digitalWrite(6, HIGH);

  aaAudio.begin(3, 0);
  aaAudio.dacBitsPerSample = 8;
  aaAudio.adcBitsPerSample = 8;
  aaAudio.setSampleRate(9000, 0);

  udp.begin(4444);
}

uint32_t timer = millis();
uint32_t packetCount = 0;


void loop() {

  aaAudio.getADC(bufferSize);
  
  uint8_t finalBuf[bufferSize * 2];

  for(int i=0; i<bufferSize; i++){
    finalBuf[i] = aaAudio.adcBuffer[i];
  }
  
  aaAudio.getADC(bufferSize);

  for(int i=bufferSize; i<bufferSize*2; i++){
    finalBuf[i] = aaAudio.adcBuffer[i - bufferSize];
  }

  sendRTPPacket(finalBuf, bufferSize * 2 );

  packetCount++;

  if(millis() - timer >= 5000){
    timer = millis();
    Serial.print("Packets/sec: ");
    Serial.print(packetCount/5.0, 2);
    Serial.print(" | Seq: ");
    Serial.print(rtp_sequence);
    Serial.print(" | TS: ");
    Serial.println(rtp_timestamp);
    packetCount = 0;
  }
}

void sendRTPPacket(uint8_t* payload, int payload_len) {
  uint8_t rtpPacket[12 + payload_len] = {0};
  
  rtpPacket[0] = 0x80;
  rtpPacket[1] = (0 << 7) | RTP_PAYLOAD_TYPE;
  
  rtpPacket[2] = (rtp_sequence >> 8) & 0xFF;
  rtpPacket[3] = rtp_sequence & 0xFF;
  rtp_sequence++;
  
  rtpPacket[4] = (rtp_timestamp >> 24) & 0xFF;
  rtpPacket[5] = (rtp_timestamp >> 16) & 0xFF;
  rtpPacket[6] = (rtp_timestamp >> 8) & 0xFF;
  rtpPacket[7] = rtp_timestamp & 0xFF;
  rtp_timestamp += SAMPLES_PER_PACKET;
  
  rtpPacket[8]  = (rtp_ssrc >> 24) & 0xFF;
  rtpPacket[9]  = (rtp_ssrc >> 16) & 0xFF;
  rtpPacket[10] = (rtp_ssrc >> 8) & 0xFF;
  rtpPacket[11] = rtp_ssrc & 0xFF;
  
  memcpy(&rtpPacket[12], payload, payload_len);
  
  udp.beginPacket(targetIP, targetPort);
  udp.write(rtpPacket, 12 + payload_len);
  udp.endPacket();
}

I'm also using this configuration file for VLC Media Player (create audio.sdp with this info and click it to start VLC):

v=0
o=- 0 0 IN IP4 10.10.1.63
s=Arduino UDP Audio
c=IN IP4 0.0.0.0
t=0 0
m=audio 5004 RTP/AVP 96
a=rtpmap:96 L8/9000/1

Basically, I'm trying to find the bottleneck here. If I run it unchecked without sampling the ADC I get 9.6 packets/second.

2

Have you attempted to measure the execution time of a single loop or any of the functions you call within the loop? That's the first step in locating a possible bottleneck.

The least invasive way involves setting a digital pin and monitoring with an oscilloscope.

3

Thanks for your reply!

I haven't exactly measured the execution times, but I've managed direct streaming to the DAC at 16-bits 44KHz, as well as SD Playback and streaming over nRF24 radios.

I'm using very similar code here, the only real addition is the UDP sending, which is what seems to be introducing the big lag.

If I remove everything except the UDP code, it still only runs at 9.6 packets a second, which is just under (1024 bytes per payload + 12 bytes header) * 9.6 packets per second = 10KB/s, pointing again to the UDP sending code:

  udp.beginPacket(targetIP, targetPort);
  udp.write(rtpPacket, 12 + payload_len);
  udp.endPacket();

If I add the audio code to actually sample the ADC, it drops to 9.2 packets per second which is in line with my audio sample rate of 9KHz.

I've also attempted to modify the ESP32 bridge code, but if I change the line suggested here to 921600 baud, the device fails to connect to WiFi: Increase bandwidth of Wifi communication · Issue #55 · arduino/uno-r4-wifi-usb-bridge · GitHub

I am very confident at this point the bottleneck is within the UDP sending, but not sure where it is internally within that or how to overcome it.

4

Did you change the baud rate in both the ESP32 firmware and in the modem code for the R4? The Modem.h file in the WiFiS3 library has a default of 115200 baud for begin(). Changing the modem.begin() in your code to modem.begin(921600) is likely not sufficient, since there are numerous calls to modem.begin() within the WiFiS3 library.

5

Thank you @david_2018 !!

I modified all the code in UNOR4USBBridge.ino as follows:


#ifdef DEBUG_AT

  SERIAL_AT.begin(500000);

  while (!SERIAL_AT);

  SERIAL_AT.println("READY");

#else

  USB.VID(0x2341);

  USB.PID(0x1002);

  USB.manufacturerName("Arduino");

  USB.productName("UNO WiFi R4 CMSIS-DAP");

  USB.firmwareVersion(U8TOBCD(FIRMWARE_MAJOR) << 8 | U8TOBCD(FIRMWARE_MINOR));

  //USB.enableDFU();

  DAP.begin();

  SERIAL_USER.onEvent(usbEventCallback);

  SERIAL_USER.enableReboot(false);

  SERIAL_USER.begin(500000);

  SERIAL_USER.setRxBufferSize(0);

  SERIAL_USER.setRxBufferSize(2048);

  SERIAL_USER_INTERNAL.setRxBufferSize(8192);

  SERIAL_USER_INTERNAL.setTxBufferSize(8192);

  SERIAL_USER_INTERNAL.begin(500000, SERIAL_8N1, 44, 43);

  SERIAL_AT.setRxBufferSize(8192);

  SERIAL_AT.setTxBufferSize(8192);

  SERIAL_AT.begin(500000, SERIAL_8N1, 6, 5);

  USB.begin();

Then I set the following in Modem.h: void begin(int badurate = 500000, int retry = 3);

I also adjusted it in my main sketch, setting all calls to 500,000 baud.

I now have it streaming at 8-bits 32Khz !!!!

In my case, I couldn't get it working reliably at 921,600 baud, but 500,000 baud worked! I will have to experiment a little now!

6

The first code I posted will work with these modifications just by adjusting the sample rates defined in the .ino file as well as the .sdp file. Here are the details for 16-bit streaming at 16KHz with the above changes:

#include <AutoAnalogAudio.h>
#include <WiFiS3.h>
#include <WiFiUdp.h>
#include "secrets.h"

char ssid[] = SECRET_SSID;
char pass[] = SECRET_PASS;
int status = WL_IDLE_STATUS;

const char* myName = "ArduinoR4";
const char* targetIP = "10.10.1.174";
const uint16_t targetPort = 5004;

const uint8_t RTP_PAYLOAD_TYPE = 96;
const uint16_t SAMPLE_RATE = 16000;
const uint16_t SAMPLES_PER_PACKET = 512;

uint16_t rtp_sequence = 0;
uint32_t rtp_timestamp = 0;
uint32_t rtp_ssrc = 0x12345678;

AutoAnalog aaAudio;
WiFiUDP udp;

#define bufferSize 256

void setup() {

  Serial.begin(500000);
  Serial1.begin(500000);

  WiFi.setHostname(myName);
  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    Serial.println("Please upgrade the firmware");
  }

  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);
    status = WiFi.begin(ssid, pass);
    delay(10000);
  }

  Serial.print("IP Address: ");
  Serial.println(WiFi.localIP());
  Serial.println("Analog Audio Begin");

  pinMode(5, OUTPUT);
  digitalWrite(5, HIGH);
  pinMode(6, OUTPUT);
  digitalWrite(6, HIGH);

  aaAudio.begin(3, 0);
  aaAudio.dacBitsPerSample = 16;
  aaAudio.adcBitsPerSample = 16;
  aaAudio.setSampleRate(16000, 0);

  udp.begin(4444);
}

uint32_t timer = millis();
uint32_t packetCount = 0;

uint8_t finalBuf[bufferSize * 2 * 2];

void loop() {

  aaAudio.getADC(bufferSize);
  for (int i = 0; i < bufferSize; i++) {
    aaAudio.adcBuffer16[i] = __REV16(aaAudio.adcBuffer16[i]);
  }

  memcpy(&finalBuf[0], (uint8_t*)aaAudio.adcBuffer16, bufferSize * 2);

  aaAudio.getADC(bufferSize);
  for (int i = 0; i < bufferSize; i++) {
    aaAudio.adcBuffer16[i] = __REV16(aaAudio.adcBuffer16[i]);
  }

  memcpy(&finalBuf[bufferSize * 2], (uint8_t*)aaAudio.adcBuffer16, bufferSize * 2);
  sendRTPPacket(finalBuf, bufferSize * 2 * 2);

  packetCount++;

  if (millis() - timer >= 5000) {
    timer = millis();
    Serial.print("Packets/sec: ");
    Serial.print(packetCount / 5.0, 2);
    Serial.print(" | Seq: ");
    Serial.print(rtp_sequence);
    Serial.print(" | TS: ");
    Serial.println(rtp_timestamp);
    packetCount = 0;
  }
}

void sendRTPPacket(uint8_t* payload, int payload_len) {

  uint8_t rtpPacket[12 + payload_len] = { 0 };

  rtpPacket[0] = 0x80;
  rtpPacket[1] = (0 << 7) | RTP_PAYLOAD_TYPE;

  rtpPacket[2] = (rtp_sequence >> 8) & 0xFF;
  rtpPacket[3] = rtp_sequence & 0xFF;
  rtp_sequence++;

  rtpPacket[4] = (rtp_timestamp >> 24) & 0xFF;
  rtpPacket[5] = (rtp_timestamp >> 16) & 0xFF;
  rtpPacket[6] = (rtp_timestamp >> 8) & 0xFF;
  rtpPacket[7] = rtp_timestamp & 0xFF;

  rtp_timestamp += SAMPLES_PER_PACKET;

  rtpPacket[8] = (rtp_ssrc >> 24) & 0xFF;
  rtpPacket[9] = (rtp_ssrc >> 16) & 0xFF;
  rtpPacket[10] = (rtp_ssrc >> 8) & 0xFF;
  rtpPacket[11] = rtp_ssrc & 0xFF;

  memcpy(&rtpPacket[12], payload, payload_len);

  udp.beginPacket(targetIP, targetPort);
  udp.write(rtpPacket, 12 + payload_len);
  udp.endPacket();
}

The associated SDP file would look like:

v=0
o=- 0 0 IN IP4 10.10.1.63
s=Arduino UDP Audio
c=IN IP4 0.0.0.0
t=0 0
m=audio 5004 RTP/AVP 96
a=rtpmap:96 L16/16000/1

Of course, the IP addresses in the .ino and .sdp file would need to be modified as well to match your sender and receiver accordingly.

7

Probably caused by the oscillator frequency being off, the UNO R4 WiFi does not have an external crystal/resonator.

8

It might be more convenient to set a const variable to the value of the baud rate (and maybe also the buffer size) at the start and use that variable in each statement where it is needed. That way, if you need to tweak those values again for any reason, they only need to be changed in one place.

Typical baud rates above 115,200 and lower then 921,600 are 230400, 460800, 576000. You may be able to push it a little more to 576000.

9

Also a good idea to make a note in the sketch about the modification to Modem.h in the WiFiS3 library, since that will get lost if the board package is updated.

10

Yeah, that seems like solid advice since I will be testing custom baud rates. At 576000, I can run 20Khz, 16-bit audio.