When starting out, you need to have a basic stock of commonly-used components in commonly-used values. Grab a few 'value' packs of resistors, ceramic capacitors, electrolytic capacitors, some 1N914 1N4148 & 1N4007 diodes, various potentiometers, common NPN & PNP transistors such as 2N3904/2N3906 or 2N2222/2N2907 or BC54X/BC55X or equivalent with their complementary and a couple of spools of enamelled wire, in different gauges, for making inductors/transformers etc.
Grab also, a bag of various coloured LED's, a couple of 555's and a couple of Op-Amps.
Salvage some old Cat5, or other solid-cored cable for hooking up on a breadboard, along with battery-snaps for power supply, or purchase an adjustable bench supply.
12V, 9V & 5V regulators (LM7812, LM7809 & LM7805, or LM340T12 & LM340T9 & LM340T5) will also come in handy.
Once you have these components and have built a few of the basic example circuits available, you will get a feel of the other components you need to add to your inventory to build more complex circuits. (Hint: - You can connect resistors in series, to create values you don't have individually.)
Bookmark a few useful links you may need to refer to frequently:
Ohm's law:
Ohm's law - Wikipedia, the free encyclopedia
LED resistor calculator:
Bread-boarding for beginners:
Breadboards
HTH.