Hi! I've been working on a project where I process audio into raw levels and output them using a breadboard R2R DAC into a speaker. I've managed to get audio playing by storing short clips in program memory, but now that's sorted, I wanted to try and see if I could process audio using python on a PC (or Raspberry Pi) and then send the raw audio level data via. USB to the Arduino Mega, thus allowing for larger files to be played.
Here's my Arduino code...
char received;
void setup()
{
Serial.begin(115200);
}
void loop()
{
while (!(Serial.available() > 0));
received = Serial.read();
Serial.write(received);
}
... and here's the python code using the pySerial module on my PC.
import serial
from time import sleep
from sys import byteorder
integers = [0, 16, 32, 128, 200, 248, 255] # Arbitrary levels.
with serial.Serial('COM3', 115200) as ser:
while True:
for integer in integers:
if ser.writable():
# Write the byte representation of this integer to serial.
ser.write(integer.to_bytes(1, byteorder, signed=False))
# Give it a second (this seems to make receiving more reliable)
sleep(0.01)
# Receive data and print it as its integer representation.
while ser.in_waiting > 0:
print(f'Received: {int.from_bytes(ser.read(), byteorder)}')
sleep(1) # PROBLEM AREA
sleep(4)
My problem is that when I change sleep(1) to sleep(0.01) and send bytes to the Arduino quickly (which would be a requirement for streaming audio), the Arduino stops sending any data back (the Tx indicator doesn't even blink). From what I can tell, reading might partly be the issue - maybe when the input buffer overflows? I'm not sure how to rectify this but keep speeds quick. Any suggestions?
You can’t send them too quickly, they are paced at 115200 bauds…
Seems you are trying to second guess timing of an asynchronous process with inserting delays, that’s not a good idea.
I proposed some code in this discussion Where I use queues on the Python side
Sorry to bother you again. I feel a little guilty about wasting your time, so let me know if you don't have anything else to suggest.
Your example was helpful in illustrating the importance of trying to do things asynchronously - and I've half-achieved what I've set out to do. The Arduino doesn't hang on execution anymore, but now transfer speeds are painfully slow - even with a high Baud rate (400,000). No errors though.
My Arduino code is still the same, but here's my revised python code.
import serial, queue
from time import sleep, perf_counter
from sys import byteorder
data = [] # After adding data, will be 14,848 bytes long.
for level in range(256):
for _ in range (15_000 // 255):
data.append(int.to_bytes(level, 1, byteorder, signed=False))
with serial.Serial('COM3', 400000) as ser:
while True:
now = perf_counter() # For recording entire data transfer time.
times = 0.0 # For recording average wait time on return of data
for byte in data:
ser.write(byte)
# print(f"Sent {byte}")
time_start = perf_counter() # Waiting for response started now.
while True:
if perf_counter() > time_start + 2: # If it takes 2 seconds to get a response, time out.
print("Timed out!")
break
elif ser.in_waiting > 0: # If we received a response,
received = ser.read()
# print(f"Received {received}")
if received == byte: # And it matches what we sent,
times += perf_counter() - time_start # Record the time it took
break # Next byte.
print(f"Completed data transfer in {perf_counter() - now} seconds")
print(f"\t- Wait avg. of {times/len(data)}")
sleep(2)
It takes ~60 seconds (give or take a second) to transfer all data (this code creates 14,848 bytes of data to transfer. It takes ~4ms to wait for a response each time, which, multiplied by 14,848, is an explanation of where all that time went. I'm sure this is an issue with using a virtual serial connection - but do you know of any way to cut this time down? Any other methods I should try, even excluding Serial connections?
Dont ack each byte, that’s costly as you make the whole thing totally synchronous.
Send a payload of say 64 bytes with a CRC and the arduino could only check the CRC for example. It would be good then to have a start payload sequence / handshake so that you can sync the emitter and receiver
Using an ESP32 rather than the mega would get you a faster cpu on the arduino side with more memory and data transfer could use WiFi for example instead of Serial and you’ll get the transport layer solved for you by TCP-IP
That could work, though I'm using the Mega (or Uno) for the ability to set pins easily for my R2R DAC. I don't have an ESP32, though if using one would be better as you say, I'm not opposed to getting one. For perspective, here's my code that plays a simple sine wave on my current board (Mega or Uno).
unsigned char sine_levels[256] = {};
unsigned char i = 0;
void setup()
{
DDRC = 255; // Set all pins on register C to output.
calculate_sine(sine_levels);
}
void calculate_sine(unsigned char* arr)
{
for (int i = 0; i<256; i++)
{
arr[i] = round((127.5*sin((2*M_PI*i)/255-(M_PI/2)))+127.5);
}
return;
}
void loop()
{
PORTC = sine_levels[i++]; // Set corresponding pins to this specific sine level.
delayMicroseconds(6);
}
I can also store short audio samples into arrays with PROGMEM and iterate through them to play audio out of the speaker. It works well, but the next step (streaming audio) is my issue. I'm working on using packets right now, but if this fails, I might consider an ESP32.
Daily check-in. Buffer idea works - just inconsistently, probably due to my shoddy craftsmanship. My python script is quite similar to as it was before, though I completely overhauled my Arduino code. There's a lot here, so feel free to skip past it.
import serial
from time import sleep, perf_counter
from sys import byteorder
with open('audio.raw', 'rb') as file:
file_data = file.read()
byte_data = bytearray(file_data)
del file_data
packetsize = 64
timeout = 2
with serial.Serial('COM3', 115200) as ser:
while True:
test = 0 // For debugging
beginning = perf_counter() // For debugging
for index in range(0, len(byte_data), packetsize):
payload = byte_data[index:index + packetsize]
ser.write(payload)
start = perf_counter()
while True:
if ser.in_waiting > 2:
ser.read_until(b'\r\n')
break
elif perf_counter() - start > timeout:
print("TIMEOUT!")
break
test += perf_counter() - start // For debugging
sleep(2)
print(f"Iteration complete.\n\t- Average wait is {test/(len(byte_data)/packetsize)} s")
print(f"\t- Total Elapsed Time {perf_counter() - beginning}")
And for the arduino:
const short BUFFER_SIZE = 63; // Includes 0 for a total of items.
// Buffer variables for handling level data, and whether these buffers are filled.
byte BUFFERS[3][BUFFER_SIZE];
bool BUFFER_STATES[3] = {false, false, false};
// Variables for handling reading into ports.
short CURRENT_READ_BUFFER = 0;
short CURRENT_READ_INDEX = 0;
// Variables for writing from serial.
short CURRENT_WRITE_BUFFER = 0;
short CURRENT_WRITE_INDEX = 0;
// Timing variable to attempt to keep audio output consistent.
unsigned long TARGET_TIME = 0;
// Some setup...
void setup()
{
// Begin a serial connection at 500,000 bauds.
Serial.begin(115200);
DDRC = 255;
// Start playing right away.
TARGET_TIME = micros();
}
// Function for reading data from serial.
void READ_DATA()
{
// While there is data available and this buffer needs data:
if (Serial.available() && !BUFFER_STATES[CURRENT_WRITE_BUFFER])
{
// Read data into the buffer at index.
BUFFERS[CURRENT_WRITE_BUFFER][CURRENT_WRITE_INDEX++] = Serial.read();
// Is this buffer full?
if (CURRENT_WRITE_INDEX > BUFFER_SIZE)
{
// Mark this write buffer as full, move to the next, and reset index.
BUFFER_STATES[CURRENT_WRITE_BUFFER] = true;
CURRENT_WRITE_BUFFER = (CURRENT_WRITE_BUFFER + 1) % 3;
CURRENT_WRITE_INDEX = 0;
// Let the serial device know we need more data!
Serial.flush();
Serial.println("OK");
}
}
}
// Function for writing audio levels to ports.
void WRITE_DATA()
{
// If it's time for the next audio level and we have data for it:
if (micros() > TARGET_TIME && BUFFER_STATES[CURRENT_READ_BUFFER])
{
// Output this level and advance the index.
PORTC = BUFFERS[CURRENT_READ_BUFFER][CURRENT_READ_INDEX++];
// Calculate next time we need to output another level.
TARGET_TIME += 64;
// If we've read all data in this buffer...
if (CURRENT_READ_INDEX > BUFFER_SIZE)
{
// Mark this read buffer as empty, move tot he next, and reset index.
BUFFER_STATES[CURRENT_READ_BUFFER] = false;
CURRENT_READ_BUFFER = (CURRENT_READ_BUFFER + 1) % 3;
CURRENT_READ_INDEX = 0;
}
}
}
// Main loop
void loop()
{
// Attempting to do these two things in quasi-parallel for minimized audio distortion.
while (true)
{
READ_DATA(); // Fill a buffer if we can.
WRITE_DATA(); // Play a level if it's time to (and we have the data for it).
}
}
The audio is still heavily distorted and timed inconsistently. My bets are that something about the serial exchange is slow and costly, causing the Mega to chew through data faster than it takes it in. Lower Baud rates are (generally) more consistent, but things are still messed up. My output sums it up quite well.
TIMEOUT!
Iteration complete.
- Average wait is 0.01931983686017632 s
- Total Elapsed Time 5.419388499984052
Iteration complete.
- Average wait is 0.0036401037929943113 s
- Total Elapsed Time 2.954235300014261
Iteration complete.
- Average wait is 0.005177830455382032 s
- Total Elapsed Time 3.0767722000018694
I'm realizing I might be a bit out of my depth - but oh well. If you have no suggestions then thanks for the help regardless! I'll update here if I figure anything out.
the challenge with streaming data is to ensure that you never starve nor over-eat.
You need to be able to output the data at a pace that is consistent with the incoming data pace. a circular buffer for example could be used, large enough to accommodate fluctuations in speed
as your python code runs on something that is likely faster than the Arduino (PC, RPi) the Arduino could be in the driver seat there to ensure the buffer does not get saturated.
