Basic questions

I just did not think it would be as difficult since each board I will be powering will be getting there power from the lipo's through the existing plug that is designed for a wall wart, so I figured if I just supplied 5v to them they would be fine as they have the step down converters on the board.

Well possibly I misunderstood or misread your needs. A drawing might be useful, but power requirements is product of both voltage and current. You stated the voltage requirements pretty clear, but the current requirements were a little unclear to me, other then the ratings of the wall warts you provided which added up to much higher then an Arduino can provide or of the external regulator you linked to.

Lefty