When driving that many LEDs per segment, you're really better off building a higher voltage supply just for the display, say 25 to 30v in your case. The current requirements will be a lot less, switching transistor selection is easier and there are some IC LED drivers that can still handle 30 volts so you may be able to keep the parts count down. In addition, you can put all the LEDs in series which makes for a lot easier implementation on the board. The idea of series/parallel arrangement of LEDs just so you can use a 5v supply doesn't sound like a lot of fun to me. I'd much rather build a higher voltage supply using a 7824 if you insist on regulation -- and this application doesn't require regulation -- rather than a really beefy 5v, multiple-amp supply to do the same job.
Dean