Well that's 5V so it looks like the LEDs are the issue - what happens if you connect them with 100 ohms series resistor direct to 5V and GND?
Most usb Ports can provide 0.5A at 5V, so running a few LEDs should be no problem.
100R should also be ok, although perhaps a little low for a red led. 220R or 330R is usually a good safe value for playing around with LEDs of any colour.
Sounds like you have faulty/damaged LEDs or a faulty/damaged Arduino.
To figure out what's damaged or not working for whatever reason, and to help you learn how electricity works, you will need a multimeter. Borrow one until your own arrives. A basic £10/$10/€10 multimeter will be fine for a beginner.