Z80 peripheral chips
First of all, the newer Z80 variants from Zilog have a much higher level of integration, and don't generally don't require the external peripheral chips. Still, the original Z80 series is a great place to start for learning about how microprocessor systems work (To learn about stand-alone microcontrollers, I'd go with a PIC, AVR or 8051). Old Z80 parts are also very inexpensive as surplus (Jameco lists a basic 2MHz Z80 for $0.50).
You could try looking for an old Xerox "bigboard" or 820-II board, these were very popular with hobbyists during the 80's, and include the Z80 and most of the peripheral chips you mentioned. Very low performance by todays standards (4Mhz clock rate), but useful as a learning tool, especially if you want to probe signals with a scope or logic analyzer to get a better understanding of what's going on.
If you want to build things from scratch, get a copy of "Z-80 Microcomputer Design Projects" by William Barden (1980, available used on Amazon for under $10). In it, he leads you through building a small Z80 based microprocessor system called the EZ-80. It doesn't include the Z80 peripheral chips, but provides a solid introduction to the Z80.
Z80 CTC - counter/timer circuit
Z80 PIO - parallel I/O interface controller
Z80 SIO - serial I/O controller
Z80 DMA - direct memory access controller
Z80 DART - Dual asynchronous receiver/transmitter
Z80 FIO - not sure about this one (FIFO maybe?)