IR would work the same way as stated by PaulS. You would need some sort of multi-directional reciever, whether it be some recievers situated at different angles, or by a rotating sensor. The human would then wear a beacon, just like they would with the XBee.
However, IR is line of sight. The beacon would have to be facing the robot, so you couldn't have a little box in your pocket or anything like that.
I like the idea about a little dish rotating on the robot. This would not be subject to line of sight restrictions, but you would have to deal with reflective signals. And I don't know if you are going to be able to get the resolution you need to actually do any tracking.
Easiest way to test it is build a little tin foil dish for the XBee, and walk around with a transmitter. See if get enough of a change in the signal strength to make you happy. If you pass in front of the dish and see a large enough jump in signal strength to program around, you may be alright.