The biggest problem stems from the way you are using the LED as a light detector. As drawn, the current that flows through the LED is the LM324's input bias current, which happens to flow out of the input on an LM324. (Bias current flows into the inputs of most other opamps). The forward drop of the LED is modulated by how much light falls on it, creating an extremely high-impedance node on the inverting input of the opamp. This will be extremely sensitive to capacitive-coupled AC signals such as from the AC power line, which will be rectified by the LED and contaminate the signal that comes from the LED illumination. To correct this, put a resistor from the inverting input to the positive supply rail. You will have to experiment with the value; make the circuit less sensitive, but still respond to the illumination change...
Using a comparator vs the opamp wont make any difference. The problems you are having is that you are using both open-loop, and the circuit is sensitive to uV of noise induced into either input causing the output to switch back and forth.
You need to add a slight amount of positive feedback to create hysteresis. Connect a several megOhm resistor from the opamp/comparator output pin to the non-inverting input. This will add enough positive feedback to prevent the continuous switching between the two states.
Putting a 10uF or so low-inductance bypass cap across the supply pins of the 324 or 339 might help.