If you want to connect an N64 controller (or a Gamecube controller - they both use a very similar protocol) to an Arduino, the code needs to be done in Assembly. The smallest time unit in the protocol is 1 microsecond (just 16 clock cycles at 16 MHz), and this precision is not possible in C. It is not the usual serial protocol so you cannot connect it to the serial pins! ALSO, the controller is 3.3V not 5V, connecting it directly to the Arduino serial pins may damage it.
The Instructables link Aleator777 posted (http://www.instructables.com/id/Use-an-Arduino-with-an-N64-controller/) seems to have code that should work (I haven't tested). Make sure you are connecting the controller properly. Use this diagram: http://www.instructables.com/id/Use-an-Arduino-with-an-N64-controller/step1/Wiring-the-controller/
Connect ground to the Arduino ground. Connect the +3.3 V line to the 3V3 line on the Arduino (the Arduino has a 3.3V regulator on board). Do not connect it to 5V. For the Signal line, plug it directly to an Arduino digital pin 2 (if you are using the Instructables code) and then connect a 1K pullup resistor between pin 2 and the 3.3V line.