Thats a great link Ron, pretty much answers his questions. I doubt there's a way to stop the solar panel both charging and supplying the arduino, and I dont see why you need to worry about that anyway? As long as your solar panel is powerful enough to fully charge the battery and power the arduino during the day, and then your battery large enough to power your circuit at night and not discharge itself too low. For that work out your required AH battery rating.
Then as an added protection, its very simple to create a load cut circuit to protect your battery from discharging too much during battery backup. For example, you can either get a voltage monitor chip and use it to trigger a high side p-mosfet, or use a simple opamp to trigger for the mosfet, or something along those lines. I'd go the voltage monitor and p-fet route, dead simple.