It's not the op amp that's sensitive to Vcc. It's the trip point as determined by the pot, since it's also connected to Vcc. If you want the trip point to be stable with a Vcc change you need a stable reference voltage for the pot.
Don't understand your V+ and V- voltages. What do they refer to? Are they the trip voltages?
What trip voltage is the pot set to?
The LM324 will only work properly when both input voltages are less the 1.5V below Vcc or 3.5V for Vcc = 5V. (Maximum input common-mode voltage from the data sheet).