The capacitor is a run capacitor, it produces a phase shift in one of the coils so the motor starts. It's called a run capacitor because it's connected to the motor continuously, a start capacitor is only connected until the motor reaches full speed.
It's true that capacitors hold there charge but most capacitors like this are fitted with bleeder resistors and it should discharge fairly quickly through the motor even if there isn't one.
EDIT:
Yes, if the capacitor has a hole in it, it means it's broken and needs replacing.