Think of it as a car radio! - the body work of the car is the ground plane, and the ground side of the radio power supply connects to that. The aerial is mounted through the ground plane, obviously insulated from it.
Your aerial should have insulators for mounting it on a metal case.
A ground plane basically provides a reflection of the aerial - so where a standard dipole aerial has two parts, one connected to the screen, and the other connected to the inner of the coax - a ground plane and whip only has one part, the whip connected to the coax inner, the screen connects to the ground plane and provides a 'reflection' of the whip, just like a lake provides a reflection of a pole sticking out of it (except a lake is an optical reflection, and a ground plane an RF reflection).
To give you an idea of how well it works, I used to have a Kenwood 2M handheld transceiver - not the tiny things you get now, but one you held in your hand and used an external microphone. The curly microphone lead was long enough to stand the transceiver on a car roof - doing so gave an increase of about 2 'S points' on the receiver meter, which made a huge difference.
Under normal use the quarterwave whip on the transceiver relied on the metal casing of the transceiver as it's ground plane, which wasn't large enough or very effective - but was easier to carry than a car roof!.