Pin 1 - VPP - To the MCLR pin on the chip. Don't forget to put a 33K or so pullup resistor on MCLR too (not critical - anything from 4.7K to 33K is ok).
Pin 2 - 5V - To VDD on your target circuit.
Pin 3 - Ground - To Ground on your target circuit
Pin 4 - PGD - To the PGD pin on the PIC
Pin 5 - PGC - To the PGC pin on the PIC
Get a 12F683, 16F88, 18F1320, 18F2550, 18F4550 or 4620. That gives you a bit of a range of different chips in different sizes with different peripherals. Other people will suggest some others probably. Whatever. Once you learn one, the rest are very similar. The PIC product line is very standardized. They mostly just have more or fewer onboard peripheral modules.
Get a 4MHz and a 20MHz crystal and a few 22pF ceramic disc capacitors. Get a dozen or so 0.1uF ceramic caps and the same number of 100uF electrolytics (and maybe a few 10uF). Some 33K (dozen), 10K (dozen), 4.7K (dozen), 1.2K (dozen), 1K (25), 330 (50) and 180 (25) ohm resistors is good to start with. If you buy these in 200-quantity or better you'll get a big price break. Get some 5mm LEDs - red, green, yellow, blue, whatever you want. A couple 10K trimpots with leads suitable for plugging into a breadboard. Some
pin header. Some
IDC connectors and an old 40-conductor IDE cable or a floppy cable to rip up and make programming cables out of. Some
2-row header pins and some
IDC male headers to make breadboard ICSP adapters out of. A **broken link removed** or two (or more) and **broken link removed** (or make your own). A roll of 30ga wire-wrap wire is useful for projects, but get a roll of something more like 26ga or 24ga for breadboard wires (30ga is too thin). A decent soldering iron and some small-guage rosin-core solder and a can of rosin paste flux too. That's a good start. What have I forgotten? Probably a few things...