Your battery is in series with your charger and Arduino... I don't think that's what you want.
I can't help but feel like you're guessing here. I think you might be in a bit over your head. Not a big deal -- you just need to start with something a little simpler and work your way up. Try just a basic solar charger first.
Tie your battery + to the Arduino input, battery - to the Arduino ground. Get that running first. Now you're powered with batteries. Then, add the solar panel...
Tie the panel - to your battery - terminal. Then, run the + output through a diode and hook that up to your battery +. Assuming your panel has (0.7v) more voltage than the battery, it will charge. Be careful that your panel does not output too much voltage! Check the specs first to make sure it won't, because there's no overvoltage protection here.
See if you can run the thing for days or weeks this way, then we can come back to adding a third power source. The panel will have to have adequate output current to charge the battery to make it through the night. You may find out your panel doesn't help much. Or it could extend the operating time indefinitely, provided nice weather. Who knows?
BTW, when you're ready to add the AC adapter, rather than have your Arduino control charging source, consider using a comparator. Using a micro for something like this is over-complicating a situation easily solved with simple analog electronics. Not that it can't be useful (or just fun), but you may have better reliability from a solution with fewer virtual "moving parts", so to speak.