At 60 Hz, the general "rule of thumb is" 1000uF per Amp of load as a filter.
For DC, of course, filtering isn't necessary. In both cases, small value caps are often used to catch spikes. Usually, less than 1uF. As far down as .0047uF, depending on the application.
As noted above, 78xx regulators drop 3 volts or so. The higher the supply voltage, the more current through the ground pin(78xx center pin) and higher the dissipated heat.
12 VAC is the RMS value, once rectified, it will be much closer to 17 volts beause you are filtering to the peak voltage.
Back in the day, when tube type transformers were still around, the 6.3 volt filiment leads would provide 8 volts(m/l) once rectified and filtered. Also available was(is?) the same rating for use on "S-100" bus micros. Linear power supplies use lots of copper.... Either was imminently suitable for a +5 volt supply. 7805's up to an amp, LM309's up to 5 amps with heat sinks.
Now, batteries.... A 9 volt MN1604 (transistor radio) battery doesn't have the ampacity to hold its' nominal voltage very long. An option for extended use is a lantern battery, either 6 volt or 12 volt. Find 'em at the BigBlueBox in the camping section. For 12 volts, you would need the regulator. For a 6 volt, 1 (or 2) 1N4006 diode in series will drop the voltage for all but the most sensitive use. If you are using standard TTL, you can operate between 4.2 and 5.1 volts, period! CMOS and LS-TTL are more forgiving. Consumer electronics.... you're on your own there, I would have to see the circuit.