The hard part about a board design as a opposed to a software program is that there is basically just one file,
Not really. In my experience hardware design tools are even more verbose than compilers about spewing intermediate files around. Just like software, a hardware design tool will use libraries and can have multiple files to build a single device.
I know quite a bit about version control, but not much about board or chip design. What are the formats? What are the specific barriers to using existing version control software?
None at all. I use subversion on both my Arduino directory and my eagle directory. With a large project, or with multiple people, I would likely put one project with both hardware and software branches into one or two repositories, but just maintaining control of the Arduino and eagle directories works for my personal stuff (where I'm mostly concerned with having my various machines at work and at home synced up, less so with the revision history).
-j