The momentary switch will connect the pin to 5V or gnd when pressed, depending on how you wired it. When it's not being pressed, what's the voltage on the pin? Totally unknown... the static charge will tend to retain the last voltage, but any noise that comes in will immediately change the voltage on the pin.
You never leave a digital input floating. You can enable the whole port's internal pullups, or you can add an external 10k resistor. Note the internal pullups MUST be used with a switch wired between the pin and gnd, not between the pin and 5V. Otherwise the pin will be read as high with the button pressed or not.