Take six serial-in-parallel-out shift registers and daisy chain them. Two connect to all of the anode connections of one color and two to the anode of the other color. The remaining two would serve as a row/column driver. You can drive them with even an 8 pin PIC. You need to send out 48 bits, each representing a color and row/column. To control the LEDs in this manor use a high clock rate so the LEDs can be refreshed enough to avoid visible flicker.
Example: First color of the first column all on, everything else off.
111111111111111100000000000000001000000000000000
Example: Second color of the third column all on, everything else off.
000000000000000011111111111111110010000000000000
Google "LED matrix"