You don't need a regulator, the phone already requires 6V which is the same voltage as the battery.
You've got current consumption, voltage and battery capacity all mixed up.
4.5Ah is the capacity of the battery.
350mA is the current consumption of the charger.
The 6V battery will be able to power the charger for 4.5/0.35 = 12 hours 50 minutes
If you put a spanner across the terminals of the battery, it will take 100s of Amps from the battery, causing it to melt.
If you connect the phone charger to the battery it'll draw just 350mA.
The only thing you might want to add is a 1A fuse to protect the cable and the battery if the charger malfunctions, creating a short circuit.