Voltage is the big thing. You should be fine with a direct connection as long as the voltage remains below 5 volts. A solar panel's open circuit voltage is pretty stable.
edit: I would use a small value resistor (200 ohms or so) between the solar panel and the Arduino pin. That way it is not enough resistance to affect the reading, but will prevent a pin blowout in the event you mistakenly set that pin as an output.