They won't like the ripple of the rectified ac supply better add a BIG cap.
I cannot see a problem most manufacturers actually recommend using PWM.
The peltiers should NOT be run without cooling on the hotside.
If you applied the 12v without cooling you would of only had a few seconds literally before the unit burnt out.
It is quite likely you have already compromised the units and they wont work as well/as long as they should do.
You cannot cool a peltier of greater max cooling (Qmax) than 100w with a fan. So yours could be fan cooled if they have individual fans. A peltier with a Qmax of more than 100w MUST be water cooled.
it is possible to run them at a higher voltage
IF the hotside is at the temperature quoted.
The problem is this:-
Peltiers work best with the maximum cooling you can provide to the hotside.
If you exceed the max input power quoted for the relevent hotside temp you will burn out the unit.
Manufacturers quote the specs at all sorts of temps the best one to use is a middling value. This is at a hotside temp of 27ºC Most Peltier users use a fan to cool the hotside with the small peltiers like yours and with an average ambient room temp and a fan on the hotside 27ºC is a nice temp to use. At 27ºC all peltiers pass 0.1277v max per couple so in your case with a standard 127 couple unit the max input at 27ºC is 0.1227 x 127 = 15.5v.
At this max input power peltiers are hideously inefficient and since at around 75% input power Peltiers can produce 80-85% of their max cooling power it is considered undesirable to power them at more than 75% power which is 11.6v. The vast majority of people run 127 couple peltiers at 12v. Some are even erronously advertised as 12v units (which of course you now know they aren't.)
So run them at 12v.
If you can find a 48v power supply run them in series, failing that get a 24v and run 2 parallel pairs in series.
At the delta you can expect for 75% input power (around 40ºC from my experience) 12v will draw just over 4.5A each.
Note your max cooling power at 12v with a delta of 40ºC will be about 20w each.
Your biggest problem will be transferring the cold from the coldside to your water it will need to very efficient.
PC enthusiasts use standard waterblocks on the coldside with another waterblock and cooling loop to a radiator on the hotside. This might be advisable for you as the radiator will stop the coolant getting too cold with a delta of 40ºC, fan cooled in average ambient room temp the coldside of the peltiers will be around 0ºC
+/- 5ºC You will also need to use a thermostat/relay. Some thermostats cant pass the cuurent so get it to operate a high current relay. You should switch the peltiers with the thermo not the power supply.
4 of your peltiers will collectively have a max cooling of around 80w but in practice it will be less than this due to thermal transfer inefficiencies. probably around 70-75w.
Note you will also have to clamp the peltiers down with a pressure of 150-300 p.s.i. This is because they naturally vibrate in use. This has the effect of creating a gap between the face of the coldside and the object being cooled. If you don't tension the TECs properly you could lose as much as 50% of your cooling i.e. just 30-40w in total for all 4 peltiers !
Unfortunately Peltier usage is not nearly as easy as the majority of people think !!!