Hi,
In addition to the other great replies here i'd like to add a little also.
Decoupling caps are used for a variety of purposes, some very specific. For general supply filtering they may be there just because they can handle the high frequency currents better than the larger electrolytic capacitors. For TTL logic families each chip or group of chips had to have a small capacitor because the output stage was partly made up of a totem pole output which would generate large spikes on the power supply lines. Placing a cap close to the chip would nip that right at the source of the problem which would reduce the effect the spike would have had on other chips, which could interfere with the timing of the circuit.
Another specific use for example is with the 555 timer ic, where the 0.01uf cap is recommended to keep noise out of the internal circuit which could cause a fluctuation in the output pulse timing. In this case the cap is not on the power supply line (although one might be good there too) but on the control voltage input, which we dont want to change due to noise pickup.
So there are a number of uses for these small capacitors, some a little more general and some very specific.