Posting your circuit could help, as we dont know whether you're using opamps/transistors or anything other than its a Colpitts.
Are you isolating the oscillator for these measurements or is there a downstream circuit attached, or something else on the power rail?
What kind of caps are you using?
Is this a PCB boarded design, or are you breadboarding or prototyping?
I've been involved in some Colpitts designs for inductive loop detection, they're usually very stable. As a side note, for the square wave generation we usually used high speed comparators (+-4ns) to minimise jitter.