A higher voltage battery hurts you if you stay with a linear regulator. You doubled the voltage, cut the current in half and have the same energy.
If you use a switching power supply it will help.
Back to the 6 volt battery. The energy between 6v and 5v is lost in heat in the low drop out regulator. about 1/6 of your total power.
If you used the 12 volt battery; it will discharge in 5 hours (using linear) because the current rating is 1/2. The voltage from 13 volts and 5 volts will be wasted in the regulator. Using a switching regulator, it will output 5V @ 100mA and take from the battery about 45mA. The switcher will move power form one voltage to another at maybe 90%. There will be less power loss.
One reason to go 12V is car accessories. I have two solar car chargers. OK, they will not charge a battery but they keep it from going down in the winter. (100mA)
Because there is no power left in the 12V battery below 10.75V, and because batteries will not charge well if totally discharged, it is not a good idea to take the battery down to 0 volts. Stop at 10.7 (and 5.4).