TOSLINK S/PDIF passthrough with optional Bluetooth A2DP: Viable for a mere software engineer?

So I will start by saying that I already solved this particular bug bear of mine, but I have done it in software, with a Raspberry Pi. This whole potential Arduino project is an attempt to take the solution I have now in Linux using mostly systems that already exist, make it a little less flexible, and solve it with a Microcontroller, purely for the learning experience, as it will be my first project.

But given that it's my first project, I want to see if it's even close to viable or biting off way more than I can chew, even knowing a bit of what I am getting into.

But anyways, onto the problem (which is equally as silly as the solution, the problem was never the point):

I want to have a box that sits behind my TV with two ports: TOSLINK in and TOSLINK out. Under normal circumstances, it's a straight passthrough. But I want to be able to connect my phone to it with Bluetooth, and mix in Bluetooth audio from my phone.

Some constraints that potentially this easier:

  • The eventual destination is going to be a dedicated DAC, then straight to my headphones. Given that, I won't be sending Dolby Digital encoded anything. Just straight dual channel PCM sent over optical. Anything else as input can blow up the moon for all I care.
  • Volume control for bluetooth can be entirely on my phone side, in fact that's preferred. The box doesn't need any notion of a volume.
  • The bluetooth audio will almost entirely be podcasts, so I am okay with mangling that a bit in the mix. My game audio should be as undistorted as humanly possible.

Some constraints that make this harder:

  • Nowhere in this pipeline do I want an analogue mixer, in fact, I want to deal with analogue audio data as little as possible. Ideally, if possible, I would write logic to mix the digital audio streams on the Arduino. If I also have to write code to match the clocks and maybe resample the Bluetooth audio to match the sampling rate of the TOSLINK input, so be it.
  • I don't want a standalone Bluetooth receiver that then plugs in via an Aux port. If I wanted that, I would do that.
  • The output TOSLINK should match excactly the input. Ideally I wouldn't need to resample the input data.

The way I picture it is:

  • There exists some board somewhere that is a A2DP Bluetooth Receiver, and can output the data in some digital format [Again I am super new here, I don't know where to call it PCM/SPDIF/I2S]
  • There exists some way to get a straight optical receiver for SFPID and either decodes it, or sends the raw bits over the wire to the Microcontroller and I deal with it there
  • I figured the mixing would all be done with Software I flash. AFAIK there must a way to resample a digital signal in software without converting it to analogue first, but I could be totally wrong.
  • There exists some way transmit the output of the microcontroller to an SFPID optical transmitter, encoded in such a way that I could deal with it in the code I write.

I know at some point in there I would have to match the clocks of the two signals, and in many ways that is the hard part, and I am honestly stumped at what step I would do that.

So what do you think? Is what I want even viable with a reasonable amount of parts, some effort, and an undergraduate degree in Computer Science?

As I have said I have implemented this in software using the magic that is PulseAudio and JACK, but I was hungry for more.

Any response appreciated, my apologies if I am super nooby here and broke some rule or made a duplicate (though I did check)!