You can power ICs from microcontroller outputs. You do have make sure that the microcontroller output can provide enough power, with a good safety margin to make sure that the voltage is very close to 0 V or 5 V depending on whether you switch the negative or positive.
It is also a very good idea to have generous suppressor capacitors.
You can provide Vss to an IC when the port pin is driven low. That is of course different from making the output high impedance. I'm not sure what you mean by "off state".