Will try with a bigger filter capacitor, and an extra one on the other side of the regulator.
I specified the transformer wrongly in my diagram, it's 18V in that it's +9/0/-9 centre tapped, and I'm just using the +9 and 0 terminals. So I should have drawn it as a 9V one. There is definitely 12V DC at most between the two sides of the rectifier.
The rectifier is correctly shown, though, isn't it? (Circuit Simulator doesn't have a symbol for the IC rectifier that I'm actually using. It's got +, - and ~ ~ clearly marked, enabling me to connect it properly).