Hello OzWold,
First off your diagram is lacking some things.
As Russlk said you need a capacitor at the output of the bridge rectifier. Without this the rectifier cannot do its job.
What is happening is that the LEDs are being fed with one half cycle of the AC voltage from the power transformer secondary and they would therefore blink ON and OFF. The (filter) capacitor will make the voltage out of the Bridge Rectifier a DC voltage so that as soon as you switch the unit ON both LEDs will light up and remain ON .... no more blinking.
The regulator needs a DC voltage on its input to function properly. Since you are feeding it with feeble half cycles it cannot output the required DC to operate your CCD CAM.
Your microphone is being fed with half cycles also and not a smoothed DC . Hence the Hum or "buzzing" sound.
Yes, in theory wire the regulator as in the diagram from the link and put a filter (electrolitic) capacitor at the rectifier output. However note that a 78LO5 is only capable of supplying 100mA and therefore will not be able to operate even your CCD CAM alone muchless the microphone and LEDs also. Use a LM7805 which is good for 1 amp.
All this should solve your "interference" problem and your CCD CAM and Microphone should function OK, assuming that they are made to operate with 5 volts.
You can check your voltages with a voltmeter. The output of the rectifier should measure about 16 volts and of course there should be 5 volts out of the regulator.
Have fun!
Trini.