If you connected them in parallel you'd get 4.2, would that do the job?
Assuming the resistance of the heater doesnt change then something like a 2.2 ohm 20 watt resistor would do the job, be warned though resistance measurements are not allways meaningfull and you could overvoltage the heater if you select the wrong value.
Also using a resistor is highly inefficient, as much energy is wasted in the resistor as the heater element, there would be no advantage of having the 2 batteries, you could so this electronically with a switch mode regulator, however I assume being a noob you dont want to dive right into building power circuits.