I don't have anything specific, but maybe start by looking for something about how op-amps work. Specifically, you need to understand how negative feedback works. A lot of audio circuits are built with op-amps or they use the same concepts.
You probably should know some basic electronics (resistors, capacitors, Ohm's Law, how to calculate power, etc). And, maybe you want to learn a little about how transistors, FETs, & MOSFETs work, although you'll probably be using ICs which can be used as a "black box" without fully-understanding the circuitry inside. (ICS can have thousands of transistors & other components inside so most of us don't understand everything about them internally.)
Transistors, FETs, and MOSFETs (or vacuum tubes) allow a small voltage or current to control a larger current or voltage so they can be used as amplifiers. An analogy would be the gas pedal in a car. Your foot simply controls a larger source of energy. ...Making a digital switch where a small voltage turns a higher power source on/off is a lot easer than making a linear analog amplifier with a controlled amount of gain.
You'll need to understand capacitors and capacitive reactance in order to understand filters. Inductors are also used in filters, but in the "audio world" you'll mostly only find them used in speaker crossovers.
In order to understand how a basic RC filter works, you need to understand how the voltage is divided between the resistance and capacitive reactance. That's one of Kirchhoff's Laws, but you can start by learning how a basic [u]voltage divider[/u] works. Kirchhoff's Laws can get a little confusing, and filters get "complicated" because of the phase shifts through a capacitor. But, if you can understand resistance, capacitive reactance, and voltage dividers, you'll have a head-start on filters.
For something like a headphone amplifier, you can look at the datasheets for headphone amplifier chips. The data sheets will give you one or more recommended circuit designs. [u]TI[/u] is a good reference.
If you're gong to be building audio circuits the biggest issue is noise! Our ears have a very wide dynamic range so we can hear a small amount of noise. You can get noise through the power supply, though the air (electro-magnetic interference), especially with high impedance or high-gain circuits, or from nearby-associated digital electronics.
For digital audio (DACs & ADCs) you first need to understand digital sampling. If you don't know how that works the [u]Audacity website[/u] has a nice easy tutorial.
You can "get started" understanding ADCs if you understand how an op-amp [u]comparator[/u] works, and that's pretty easy once you understand that an open-loop op-amp is a differential amplifier with almost infinite gain.
And, if you understand voltage dividers you can make a [u]resistor ladder DAC[/u]. (Real-world audio ADCs & DACs are more complicated.)