It's easy to build it all on the same board, you will just have longer wires? I don't get that specification...
You could have count up held high then when you push a button it increments, and when you press count down it ties count down low and also sends a pulse..just an idea
Read through this
The Seven-Segment LED Driver
When is this due?
1. You could modify or use a scoreboard from an air hockey game.
2. You could locate ( good luck!) archive issues of Electronics Hobbyist for a scoreboard game that does not use counters. One of my 1980 issues has this article that works as you describe your game plan. It uses SCRs and flip-flops.