Use a Schottky diode like 1N5820. They have less than 0.4V voltage drop. Which should be OK for your application.