How about a HEX open collector inverter for each card. You could tie the inputs to a DIP switch to set the address. 4 gates for the address, 1 for the button and 1 spare.
On the input side, use a pull up to VCC.
The issue that you will have is when two buttons are pressed simultaneously, you get giberish.
If you use a priority encoder, 10 positions could be encoded into BCD AND two buttons that are pressed at once will have definate result. I belive the highest number wins.
Now, you could for instance have reset inputs for the cards, so when simultaneous buttons are pressed, they would set appropriate flip-flops on the card. you then may need a way to reset the individual bits.
If simultaneous presses are not an issue, then your OK.