You can reduce the "interference" (if there is any) by reducing the resistor value to as low as 470 ohms, and the capacitor can be increased to hundreds of micorfarads without hurting the chip or the operation of the circuit. If you actually have interference problems after that, you can build the "guaranteed to survive lightning strikes" circuit.
C'mon guys. This is basic light bulbs and batteries, not mil spec hospital equipment.
The only thing to be ware of, is that when the power is turned on, the input to the logic gate will be low so the circuit will think the button is pressed every time it's switched on. This might not be a problem, depending on the rest of the circuit.