Internet radio playback becomes choppy after long duty cycle, why?

Other Hardware Audio
1

Video of behavior: https://youtu.be/XZqCWBApdnQ

I've got an ESP32/VS1053 internet streamer installed in an old radio. The Arduino circuit is powered by an Apple 5V cube charger, which is tied into mains before the radio's power switch. This means the Arduino section is always running (if the radio is plugged in), and the power switch just controls the original radio's amplifier circuit.

This usually works, but once or twice a week I'll turn the radio on in the morning (the Arduino will have been streaming all night) and the sound will be slow and choppy. Oddly, a single station out of twelve is never affected. Nothing special about it that I can see:

Edit: the immune station is actually 64kbps, even though the URL suggests a 128 bitrate. All other stations are 128+, so that's one thing that sets it apart.

Immune to bug

// 7 - 94FM "95B" Auckland
char *host7 = "streams.95bfm.com";
char *path7 = "/stream128";
int httpPort7 = 80;
char *httpver7 = "HTTP/1.1";

Not immune

// 1 - 88FM KXLU
char *host1 = "kxlu.streamguys1.com";
char *path1 = "/kxlu-hi";
int httpPort1 = 80;
char *httpver1 = "HTTP/1.1";

What I've tried

  • Reset the ESP32 via pins
  • Unpluged the radio at the wall
  • Wifi strength: I've now got it connecting to a specific AP and I know the signal is strong. There are never macro wifi issues for other devices when this behavior is happening.
  • Edit: Restart gateway AP, local AP, router
  • Gave Arduino fixed IP
  • Switched from Cloudflare to Google Public DNS, with and without IPv6
  • Added <time.h> to sync time on Arduino load (I think)

What works?

  • Most often, unplugging the USB cord directly from the ESP's port. Doesn't always work though.

Questions

  1. Does the sound of the audio point to a specific fault?
  2. Is the 24/7 duty cycle the cause, or just making the bug surface more often?
  3. If duty cycle is the cause, is the hardware or software degrading over the cycle?
  4. Where I should start to troubleshoot this?

github repo

////////////////////////////////////
// Internet Radio w/ Analog Tuner //
////////////////////////////////////

// Ring Buffer Version
// V2 - Add pot check into loop
// V2.1 - Add station check into loop
// V2.2 - Change station with map instead of raw pot values, remove setup connect

#include <Arduino.h>
#include <VS1053.h>
#include <WiFi.h>
#include <config.h>
#define VS1053_CS 5
#define VS1053_DCS 16
#define VS1053_DREQ 4
#define VOLUME 100
#define VS_BUFF_SIZE 32
#define RING_BUF_SIZE 32000

VS1053 player(VS1053_CS, VS1053_DCS, VS1053_DREQ);
WiFiClient client;

// wifi credentials defined in Config.h
const char *ssid = SSID;
const char *password = SSID_PASS;
const uint8_t bssid[6] = {ACCESS_POINT}; // this can be removed if you aren't targeting a specific access point/router

// 1 - 88FM KXLU
char *host1 = "kxlu.streamguys1.com";
char *path1 = "/kxlu-hi";
int httpPort1 = 80;
char *httpver1 = "HTTP/1.1"; // 1.1 is default, but some streams work better in 1.0 (not chunked)
// 2 — Deadzone
// 3 - 90FM KEXP
char *host3 = "kexp-mp3-128.streamguys1.com";
char *path3 = "/kexp128.mp3";
int httpPort3 = 80;
char *httpver3 = "HTTP/1.1";
// 4 — Deadzone
// 5 - 92FM KUSF
char *host5 = "104.236.145.45";
char *path5 = "/stream";
int httpPort5 = 8000;
char *httpver5 = "HTTP/1.1";
// 6 — Deadzone
// 7 - 94 "95B" Auckland
char *host7 = "streams.95bfm.com";
char *path7 = "/stream128";
int httpPort7 = 80;
char *httpver7 = "HTTP/1.1";
// 8 — Deadzone
// 9 - 96 WNYU
char *host9 = "cinema.acs.its.nyu.edu";
char *path9 = "/wnyu128.mp3";
int httpPort9 = 8000;
char *httpver9 = "HTTP/1.1";
// 10 — Deadzone
// 11 - 98 KFJC Los Altos Hills
char *host11 = "netcast.kfjc.org";
char *path11 = "/kfjc-320k-aac";
int httpPort11 = 80;
char *httpver11 = "HTTP/1.0"; // 1.1 caused stuttering
// 12 - Deadzone
// 13 - 100 KSCU Santa Clara
char *host13 = "kscu.streamguys1.com";
char *path13 = "/live?kscu-site=";
int httpPort13 = 80;
char *httpver13 = "HTTP/1.1";
// 14 - Deadzone
// 15 - 102 Radio K Minneapolis
char *host15 = "radiokstreams.cce.umn.edu";
char *path15 = "/";
int httpPort15 = 8128;
char *httpver15 = "HTTP/1.1";
// 16 - Deadzone
// 17 - 104 Corridos Y No Fantasias
char *host17 = "stream-150.zeno.fm";
char *path17 = "/7xv04r3wzp8uv?zs=IJNAlzc7Sa-y6rlnOKJQ2Q";
int httpPort17 = 80;
char *httpver17 = "HTTP/1.1";
// 18 - Deadzone
// 19 - 106 WVFS Tallahassee
char *host19 = "voice.wvfs.fsu.edu";
char *path19 = "/stream";
int httpPort19 = 8000;
char *httpver19 = "HTTP/1.1";
// 20 - Deadzone
// 21 - 108 Psyched! SF
char *host21 = "us3.internet-radio.com";
char *path21 = "/proxy/psychedradiosf/stream";
int httpPort21 = 80;
char *httpver21 = "HTTP/1.0"; // 1.1 caused stuttering
// 22 - Deadzone
// 23 - 110 KZSC Santa Cruz
char *host23 = "kzscfms1-geckohost.radioca.st";
char *path23 = "/kzschigh";
int httpPort23 = 80;
char *httpver23 = "HTTP/1.0"; // 1.1 caused stuttering

// pot & station mapping
const int potPin = 34;    // potentiometer is connected to GPIO 34 (Analog ADC1/6)
int potValue;             // reading from the pot (0 - 4095)
int tallyCount = 0;       // counts 10 pot readings before we take an average
int mappedTally = 0;      // a container for the 10 readings pre-average
int activeStation;        // can be any odd number from 1-11. even nums are deadzone
int previousStation = -9; // impossible value makes sure it runs through connection flow first time

// haptic
// const int hapticPin = 21; // vibration motor board is controlled by GPIO 21

unsigned long logLoopCounter = 0;

uint16_t rcount = 0;
uint8_t *ringbuf;
uint16_t rbwindex = 0;
uint8_t *mp3buff;
uint16_t rbrindex = RING_BUF_SIZE - 1;
uint8_t netbuff[1024];

///////////////////////////
// Ring Buffer Functions //
///////////////////////////

void putring(uint8_t b)
{
  *(ringbuf + rbwindex) = b;
  if (++rbwindex == RING_BUF_SIZE)
  {
    rbwindex = 0;
  }
  rcount++;
}
uint8_t getring()
{
  if (++rbrindex == RING_BUF_SIZE)
  {
    rbrindex = 0;
  }
  rcount--;
  return *(ringbuf + rbrindex);
}

void playRing(uint8_t b)
{
  static int bufcnt = 0;
  mp3buff[bufcnt++] = b;
  if (bufcnt == sizeof(mp3buff))
  {
    player.playChunk(mp3buff, bufcnt);
    bufcnt = 0;
  }
}

////////////////
// Void Setup //
////////////////

void setup()
{
  // Serial.begin(115200);
  // pinMode(hapticPin, OUTPUT); // define pin as output
  mp3buff = (uint8_t *)malloc(VS_BUFF_SIZE);
  ringbuf = (uint8_t *)malloc(RING_BUF_SIZE);
  SPI.begin();
  player.begin();
  player.switchToMp3Mode();
  player.setVolume(VOLUME);
  WiFi.begin(ssid, password, 0, bssid);
  /*while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }*/
}

///////////////
// Void Loop //
///////////////

void loop()
{
  uint32_t maxfilechunk;
  unsigned long nowMills = millis();
  int resd = 0;
  int cntr = 0;

  potValue = analogRead(potPin);
  mappedTally = mappedTally + map(potValue, 656, 3480, 1, 23);
  tallyCount++;
  if (tallyCount == 10) // after 10 loops, recalculate activeStation
  {
    activeStation = mappedTally / 10;
    tallyCount = 0;
    mappedTally = 0;
  }

  if ((!client.connected() || (activeStation != previousStation)) && ((activeStation % 2) != 0)) // if (disconnected OR new station) AND (odd station)
  {
    if (activeStation == 1) // 1
    {
      // Serial.println("Loop connect 1");
      if (client.connect(host1, httpPort1))
      {
        client.print(String("GET ") + path1 + " " + httpver1 + "\r\nHost: " + host1 + "\r\nConnection: close\r\n\r\n");
        // digitalWrite(hapticPin, HIGH); // vibrate
        // delay(200);
        // digitalWrite(hapticPin, LOW); // stop
      }
    }
    else if (activeStation == 3) // 3
    {
      // Serial.println("Loop connect 3");
      if (client.connect(host3, httpPort3))
      {
        client.print(String("GET ") + path3 + " " + httpver3 + "\r\nHost: " + host3 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 5) // 5
    {
      // Serial.println("Loop connect 5");
      if (client.connect(host5, httpPort5))
      {
        client.print(String("GET ") + path5 + " " + httpver5 + "\r\nHost: " + host5 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 7) // 7
    {
      // Serial.println("Loop connect 7");
      if (client.connect(host7, httpPort7))
      {
        client.print(String("GET ") + path7 + " " + httpver7 + "\r\nHost: " + host7 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 9) // 9
    {
      // Serial.println("Loop connect 9");
      if (client.connect(host9, httpPort9))
      {
        client.print(String("GET ") + path9 + " " + httpver9 + "\r\nHost: " + host9 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 11) // 11
    {
      // Serial.println("Loop connect 11");
      if (client.connect(host11, httpPort11))
      {
        client.print(String("GET ") + path11 + " " + httpver11 + "\r\nHost: " + host11 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 13) // 13
    {
      // Serial.println("Loop connect 13");
      if (client.connect(host13, httpPort13))
      {
        client.print(String("GET ") + path13 + " " + httpver13 + "\r\nHost: " + host13 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 15) // 15
    {
      // Serial.println("Loop connect 15");
      if (client.connect(host15, httpPort15))
      {
        client.print(String("GET ") + path15 + " " + httpver15 + "\r\nHost: " + host15 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 17) // 17
    {
      // Serial.println("Loop connect 17");
      if (client.connect(host17, httpPort17))
      {
        client.print(String("GET ") + path17 + " " + httpver17 + "\r\nHost: " + host17 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 19) // 19
    {
      // Serial.println("Loop connect 19");
      if (client.connect(host19, httpPort19))
      {
        client.print(String("GET ") + path19 + " " + httpver19 + "\r\nHost: " + host19 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 21) // 21
    {
      // Serial.println("Loop connect 21");
      if (client.connect(host21, httpPort21))
      {
        client.print(String("GET ") + path21 + " " + httpver21 + "\r\nHost: " + host21 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else if (activeStation == 23) // 23
    {
      // Serial.println("Loop connect 23");
      if (client.connect(host23, httpPort23))
      {
        client.print(String("GET ") + path23 + " " + httpver23 + "\r\nHost: " + host23 + "\r\nConnection: close\r\n\r\n");
      }
    }
    else
    {
      // nothing
    }
    previousStation = activeStation;
  }

  maxfilechunk = client.available();
  if (maxfilechunk > 0)
  {
    if (maxfilechunk > 1024)
    {
      maxfilechunk = 1024;
    }
    if ((RING_BUF_SIZE - rcount) > maxfilechunk)
    {
      resd = client.read(netbuff, maxfilechunk);
      while ((cntr < resd))
      {
        putring(netbuff[cntr++]);
      }
    }
    yield();
  }
  while (rcount && (player.data_request()))
  {
    playRing(getring());
  }
  if ((nowMills - logLoopCounter) >= 500)
  {
    // Serial.printf("Buffer: %d%%\r", rcount * 100 / RING_BUF_SIZE);
    logLoopCounter = nowMills;
  }
}

IMG_2498 2
IMG_2498 21920×1440 326 KB

2

Try to simply power down your router, switch, and modem if there is one. Restart it and see if it clears up.

3

Are you saying to do this every time the fault surfaces?

Or try it now when the radio is operating correctly, to prevent it from happening again?

4

Not every time, If it clears it up it should last a while, depending on the actual use of your system.

5

I'll give it a go next time the sound goes wonky.

What specifically do you think is happening that resetting the network would fix? I'd assumed the wifi was a fixed variable in this situation, since it resolves itself without me making any networking changes.

6

I am not sure but something fills up in the router, probably the routing table and it is doing a lot of thrashing. Resetting it cleared all of that out.

7

Can confirm that restarting networking equipment doesn't help in my case. Got the choppy audio today. First I restarting the AP that my radio is programmed to connect to, no change. Then the gateway AP (both are eero 6 pro), no change. Then the modem. Choppy audio persisted.

There also seems to be a new (worse) level to this symptom, where it will play nothing at all. That's where it started today, yesterday too. Unplugging and plugging the radio got it back to playing choppy audio. Then unplugging the USB cable from the ESP32 directly got it back to playing clean audio.

There's a pattern emerging: removing the USB cable from the ESP32 is most likely to resolve the issue. It doesn't always work. I don't see how there's any difference between the USB port and the wall plug, circuit-wise. Unplugging from the wall also removed power at the USB brick and thus the USB port. Maybe touching the ESP discharges something, or...?

8

One thing we possibly have not looked at is signal strength/quality. Is it possible your chassis is blocking the signal. Try moving it closer to the wifi and rotating a few different directions. There is code available that will allow you to measure signal strength and monitor all of the signals available. I use it on my ESP8266. I found several of my neighbor's WiFi signals.

9

I've seen packets with the wrong time stamp do the same thing in PC's. Maybe try a static IP and update the system time???

10

The eero UI shows signal strength for each device in the network, always full bars for the radio. The AP and radio sit directly next to each other on the bench, which is not necessarily ideal for mesh, but the AP used to live across the room, no difference in the symptoms.

I specified (in code) the BSSID for this nearby router because the arduino was always connecting to my gateway eero in another building. That had weak signal strength (two bars in eero UI), and behaved differently. It would take longer to connect to a stream, then stop playing, reconnect, on a loop.

(Edit: maybe reconnect is the wrong word, it uses a ring buffer so it's probably the buffer emptying out? I don't understand this part of the code very well)

The chassis is wood, but the frame stays out most of the time anyways since I'm having to disconnect the usb cable.

11

I can try a static IP. The system time part is out of my depth, can you explain? I see this example online for calling NTP from a server, but what do I do with the output?

#include "time.h"
//
const long  gmtOffset_sec = 3600;
const int   daylightOffset_sec = 3600;
const char* ntpServer = "north-america.pool.ntp.org";
//init and get the time
configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);

also is this time stamp error something that would clear itself up after several restarts, like I'm experiencing?

12

I believe that line is what does the "work". It sets the time so that your uC packets and the router packets are relatively aligned so the router doesn't think your packets are from 8 years ago or whatever. Timestamp errors are weird and can do all sorts of things, including intermittently working or not, with or without restarts. You could also be getting continuous DHCP renewals if the time is off too.

1 Like
13

Copy that, gonna give this a go

14

Unfortunately, the configTime fix didn't work. Setting a static IP for this device in eero didn't work either.

Today when I got the choppy audio I tried the usual: unplugging at the wall, resetting the arduino via pins, then unplugging the USB at the ESP32 port. Again, that last one worked. There has to be something going on with this specific action. Does anything happen mechanically on an Arduino when the USB port is physically connected/disconnected? Anything that would be different than removing 120V from the entire radio?

The single working station still confuses me as well. I learned that this station is streaming at 64 kb/s, even though the URL suggests a 128 bitrate. All the other stations are 128+. Is there something here? What fault would allow the 64 station to continue working while the 128/320 streams go choppy?

15

Have you considered your ISP may be throttling your service after so many megabits?

16

I've got gigabit service. Would a single radio stream going overnight really make a dent?

I'm always using my work computer while these streaming symptoms are happening, primarily working in Figma which is a cloud tool. When the radio is choppy I just listen to the same stream on my computer, always works fine. Would I not notice throttling there too?

If I run a speedtest while the symptom is present, that should rule out throttling right?

17

Just got the bug, so I ran a speedtest on the gateway router. Getting 938 Mbps there. On the AP in my shop that the radio connects to I'm getting a little over 200 Mbps which is disappointing but fairly normal.

18

Throttling is likely "by stream" to a single device.

19

Interesting, I don't know much about networking obviously, had no idea the ISP had that level of visibility into local IPs.

Any way to test this?

20

It may be happening in one of the many intermediate points in the data's travel. You do know that ALL, ALL, internet data, including all your streaming radio stuff, goes through one of several Government monitoring locations. Everyone on the internet in the Northwest, goes through a system in Seattle to look for who-knows-what. Any one of the intermediate locations may be the one doing the throttling.