I think there are some circuits here on this site that are very close to what you want. What you describe can easily be done by a single counter chip. You basically have 3 states: 4 LEDs on, 2 LEDs on, all LEDs off. You can use a 3-stage counter to cycle through these states.
So you'd need at least 2 chips, the counter, plus another chip to debounce the pushbutton switch. (Look up "debouncing" on this site and you'll see lots of posts on the subject.)
It's a pretty simple circuit, but unfortunately I don't have a schematic to give you. I'm sure someone else here can, though.