Note that with a real-world switch, the contacts will bounce, opening and closing a few times within a millisecond or two.
If you are trying to design a practical circuit that you can later build, you have to allow for that.
One simple approach is to use a D type flip flop, with the /Q signal connected back to D by a series resistor, and with a small capacitor to ground from D.
Something like a 220K and 0.1uF should do with a CMOS D type; that delays the signal at D a few milliseconds so switch bounce does not cause it to keep toggling, but another press after that delay effect will cause another toggle.
The switch and its pullup or pulldown still connect to clock input.