6V exactly? It's pretty easy to find a regulator IC to do the job but the closest number to 6V is 5V (used by electronics). Perhaps a zener diode to clamp the voltage to 6V.
Wasting 6v at 4A (24W) in a linear regulator is a bit excessive. By the time you've added in heatsink costs etc, you may find the the previously suggested buck regulator a much better option. This achieves high efficiency and should not waste much power.
efficiency is only part of the issue. Dr. Em is correct though a bit understated on the impact of 24W dissipation. A linear would require a rather large heatsink and decent ventilation or it will cook itself to death. when all is said and done, a buck regulator will run dramatically cooler and may even cost less than a linear+heatsink.