How is it built/wired? (PCB, Strip-board,Plug-board?)
You likely have a ground loop where the relay current (which flows into the ground path out of the emitter of the switching transistor) produces a voltage drop across some part of the wiring between the opamp and the thermistor.
From just a quick glance at your schematic, it appears your comparator has no hysteresis and may be false triggering from either some noise on the input or some other fluxuation of the input signal. I'll do a quick sim and get back to you.
Unless I've drawn it wrong somewhere, I don't believe the comparator is set up to function at all as expected. I ran the sim and stepped the inverting input at 7 levels from 4.5V to 7.5V and got 4 different output states. I've attached the sim.
I have been over it several time looking for drawing errors, but can't find any. The symptom described could be from the changing resistance of the sensor in conjunction with the configuration.
I forgot to note the input Vs. output levels:
4.5V in = 12V out
5.0V in = 4.1V out
5.5V in = 8.75V out
6-7.5V = 0V out