The problem with a simple linear regulator is getting 2 amps over the entire voltage range (or vice versa). The unregulated supply will have to be around 25 volts, so at 1 volt and 2 amps, the regulator has to dissipate 48 watts. The usual solution to this problem is a switching regulator, but switchers generally have some residual high frequency noise on their outputs.
I once played around with a design that had 2 SCRs in the bridge (or center tapped) rectifier, and a tracking circuit that phase-modulated the SCRs to keep the voltage across the linear regulator at about 5 volts. I simulated it, but never built it, but I have no doubt it would work. You can also use a similar scheme, but with a tracking switching preregulator, if you need really clean (read linear regulator) DC. Keeping switching noise out of the output still requires careful attention to grounding and shielding.