Kris:
In some designs you may find a tantalum electrolytic paralled with a ceramic and possible paralleled with a metalized polyester. It's done that way for various reasons, but briefly a capacitor may have a model where there is a lead inductance, series resistance, parallel resistance and an ideal capacitance. Those parameters and their tolerences change with things such as temperature, frequency etc. An electrolytic cap used in a switching power supply is going to different than one used for a line operated power supply. If the latter is used in a switching supply, it may heat up and fail. Tantalum electrolytics do not tolerate overvoltages well at all, but they have an extremely low ESR (Effective Series resistance) and have a large capacitance density. Generally you cannot make electrolytics with say a 1% tolerance. Not all capacitors can be made with the same capacitance or voltage tolerance.
Metalized Polyester is a good compromise for coupling caps on the order of 1 uf or less. Electrolytics are used in the cheap stuff. More exotic capacitor types are used by purist audiophiles. Those will generally be used in instances where tolerances matter such as frequency filters.
Decoupling is where you'll find multiple types. These caps are located very close to the power supply pins of a device. a logic gate draws a brief pulse of power when it switches and an electrolytic won't respond fast enough no matter where it is (at the power pin or 6" away). Ceramics have been used in this application.
Microwave circuits will use surface mount devices because the lead resistance gets in the way. An Electrolytic would be totally inapproprite in this application.
There are variables such as cost, lifetime, reliabilty and appropriateness for the technology.