The button on the case is momentary, but it's not connected to the power supply, it's connected to the motherboard, which is receiving a standby 5 volt line to power the electronics which actually control the power supply turn on line. The motherboard of a modern PC is never actually off, unless disconnected from the wall, sometimes there's a bypass switch on the back of the power supply but not always. The momentary on, and push and hold to turn off is the standard method motherboards use to control the switch that goes to that green line, and is not part of the ATX standard. Look it up.
As I said, I'm not guessing here, I've done this, on more than one power supply.