As the supply is typically rectified mains, the rectifiers stop the voltage being clamped by the source.
I guess Prasannan82 has a capacitor rated at less than the supply used for his constant current source. His problem with residual charge is caused by dielectric absorption. The best capacitors for this application were polycarbonate, but they are no longer made (I think the plastic film was single source and the supplier stopped making it). The next best options are Teflon (PTFE very rare for capacitors), polystyrene and polypropylene.
Any type of electrolytic will be very poor, even if you do leave it shorted for a week!