One thing I don't like is how glitchy the square-wave input is to the gate:
View attachment 89729
But the good news is that adding a load (remember: note to self: always model a load!) quickly stabilizes the output. Plus the current source can easily drive a 5k load (it'll go down to 1K, albeit with some distortion).
I did change R8 to 1K and R5 to 22K to try to isolate the current sink from the oscillator, which helped somewhat. Actually, that spike on the rising edge doesn't seem to make much difference anyhow.
Good that it worked.
Today I felt inclined to tweaking. Now no more weird scheme; just +Vcc.
I provided my own (simpler) astable driving the switching. To get the switching working properly with a single supply I needed to reference both current sources to a point halfway between ground and Vcc (R9 & R10 divider on the right).
If the second it takes to stabilize seems too long (as you say, is not), play with R9 & R10. They control that part upon start.
My current suggestion, astable included, is simpler and works.