What's your location?
What sort of electronics are you going to be doing?
For a basic starter kit I'd recommend the following:
An E12 set of carbon film resistors, 10R to 1M. I recommend buying extra common values, i.e. E3 values between 100R and 100k.
An E3 set of capacitors, 10pF to 1000µF. I recommend buying extra 100nF capacitors, you'll come across these all the time.
The above can normally be bought as kits.
Small signal transistors, e.g. PN2222A (NPN), PN2907 (PNP).
Some MOSFETs, e.g. IRL540.
Diodes: 1N4001 (rectifier), 1N4148 (small signal), BAT85 (Schottky small signal), 1N5821 (Schottky barrier).
Some cheap wall plug mains adaptors, look in charity shops or trash pick from the dump.
Regulator ICs: start with LM7805 and LM317, not so important if the wall plugs you buy are regulated.
Common op-amps, e.g. TL072, TL074, LM358, LM324.
A small audio amp IC e.g. LM386.
Comparators, e.g. LM393, LM339, LM311.
Small 8Ω speakers, look for old radios/TVs at the local dump.
Some CMOS glue logic, i.e. CD4011, 74HC01, 74HC04, etc. not needed much these days but still useful for simple circuits.
Are you interested in micro-controllers?
Buy a cheap programmer ans some PICs.
See Nigel's tutorials.
Another good PIC site.
Gooligum Electronics
Essential tools:
A set of screwdirvers
Soldering iron
Desoldering pump
Multimeter
Breadboard
Also nice:
Oscilloscope - look on ebay
Signal generator - ebay again or build your own.
Benchtop power supply - build your own.
Consumables:
Solder
Stripboard
Tinned copper wire
Also nice:
Etching kit.