There are plenty of cheap and good quality wireless web cams with remote steering control. A few of those dotted around the house would get you to 90% of your requirement for zero effort. The ones I've used support dynamic dns, motion detection/alerting via several methods and seem ideal for what you're asking for. I know you said you're doing this for fun, but you might want to get the simple cheap solution working anyway so you have something to watch while you develop your own rover based solution. They are available with low light sensitivity and IR illumination which makes them especially useful for monitoring a house which is not occupied.
The standard cameras provide an HTTP interface so can be controlled and viewed via a web browser - no special client software needed. One of the ones I use did also come with a client that managed a bunch of cameras and presented them in a grid on the screen with zooming and recording and so on, but that's just bells and whistles and a web browser is all you need.
If you want stereo vision then you can't simply take a pair of steerable cameras - you would need them to be mounted on a common steerable head. One option would be to canibalise the steering mechanism from a standard cam and figure out how to attach a pair of cameras in place of the single standard one. That should just be a matter of cutting and gluing. That gives you web based steering as well as the usual controls for brightness etc. You will almost certainly find that the camera itself hosts a web site that presents the live camera feed in the same page as the widgets needed to configure and control it so the whole thing will be very user friendly.
You do also have the option of making your own steerable camera mount from scratch and using RC servos to operate it - more satisfying but a huge amount of extra work. The Arduino will control these without any problem, but now you need your Arduino to take the place of the standard controller and provide a web service or UI to control it, and if you want to integrate that with the live video feed then you will need to make your own mashup to do it.
For the rover, all you would need is a wifi interface. If you're going to put the cameras on the same rover then you might consider mounting a wifi router on the rover and using ethernet connections to the Arduino and cameras - this simplifies things considerably and should make your rover much more reliable - if it can cope with lugging a router box around as well as everything else.
Note that if this rover is going to be wireless then it will need to be battery powered - the power consumption of a rover and pair of wireless cameras will be considerable so you will either need to have some way to recharge it, or compromise between a huge battery or a very short battery life.
While you're looking into stereo display headsets, you might consider using a pair of cameras much further apart than your eyes to give you an artificial sense of scale. I've never done it myself but I've read of some very cool looking projects using this approach to look at landscapes and cloudscapes.