These days, a high-capacitance, ultra-low ESR MLCC cap is QUITE cheap for low voltage.
You can get like 10uF-22uF pretty cheap and tiny. There's generally no need for 100uF in combatting electrical noise. Noise will not be sustained that long.
In some cases you can cause latchup by inducing >Vdd on any input pin, so don't neglect this possibility. And you can do it while it's an OUTPUT, too. If the pin's low and you drive current in from an external source, the pin drive tends to ground it out. If the pin's high and you drive current into it, it jumps to the Vdd rail and that can cause latchup.
Leading to the next point not everyone understands: I/O pin protection against overvoltage consists of shunting current to the Vdd rail, and the Vdd input pin, which will actually raise Vdd on the board trace. A regulator mainly has the drive to INCREASE voltage on the output. It often has very little ability to shunt the excess voltage and LOWER the voltage on the output rail. Anything on the board which shunts excess voltage onto the Vdd rail has the potential to cause latchup in anything which uses the Vdd rail.