Some calcs involved.
A battery wont do it. U need some serious fuel. U must construct a fuel cell. Paraffin wax packs 42 kJ per gram at .8g/mL. A pack of cards is about 71mL or 56g of wax.
The Specific Heat Cap of water is 4.2 KJ/Kg/deg C. A litre of water is 1 kG so u need 8.4 KJ per deg C. If your ambient is 25 deg C , the delta Temp is 37-25 = 12 Deg C.
Which is 8.4 * 12 = 101 KJ of energy required. Estimating heat transfer at 70% we have 100/70 *101 = 144 KJ required.
To get 144 KJ u need to burn 144/42 = 3.5g of wax. this uses3.5/56 = about 6.25% of the pack of cards. Double this to provide for the flame & air intake leaves u with about 87% of the space for the valve and valve controls. Although the valve is a 12V device , it will pull with a fresh 9VDC alkaline source.
The 9VDC batt occupies about 18 mL of space.
The valve is 3.2cm x 1.9cm x 6.2 cm = 38 mL volume
So far we have 9 ml for the wax/flame, 18mL for the Duracell 9V, 38mL for the valve.
That's about 65mL. Add in another 2mL for an SMD LM358 dual opamp, thermistor, flywheel diode, & MOSFET driver and a couple resistors and you have your control for the valve.
Total 67ml, or about 94% the size of a pack of cards. I'd use the remainingg space to apply thermal insulation to keep the heat transfer efficiency up.
Setup device so that multiple wicks from the wax heat the flow valve. Heat the aluminium part of the valve that houses the flow channels. U can use the tiny NPT fittings supplied. The valve is good to 122 Deg F
Use a valve like this one:
EBC Electronic Boost Control Solenoid Kit DIYAutoTune.com
It is a 3 port valve, so u can throttle the valve using an opamp to achieve a flow of 37 deg C. Basically, set it up so that if the outlet water is under the tgt temp, u throttle the valve towards recirculating the water to the source.
Of course, the valve is entirely off (recirc) during the first heating phase until the entire 2L source has reached perhaps 36 Deg C. Then the valve is opened fully to flow to the target. Remember the fuel is sized to run out when the desired temp is achieved. So no chance of overheating.
This makes good background reading.
https://www.sciencebyjones.com/heat_of_combustion.htm