But I'm not sure about the power supply. I tested it with the blink example code and tried to just connect a 3V battery but that didn't work.
Do I need exactly 5V and how can I get that and do I need a tool which guarantees that there's a constant flow of 5V?
The ATmega328 will run on any voltage between 2 and 5V, but you need an 8 MHz crystal for voltages lower than about 4V, and choose the appropriate board in the Arduino IDE (I use 3.3V Pro Mini).
When using the bare chip, you really need to study its datasheet - that will answer all questions about what voltages it requires, what speeds you can run it at, etc ...
Note that the Arduino UNO R3 actually uses an atmega328P:
Not a stupid question, but you can't rely on the wire colors to be as advertised. Those are common, but I've seen others. And for a USB charge-only cable, there may be only two wires.
If you don't have a multimeter, get one, and check for 5V. The data wires, if the cable is connected to a computer, will carry about half that with respect to common ground.
A multimeter is an absolutely invaluable tool for this hobby. This one is fine for starters.