I would just put the LED(s) and detector(s) in separate black tubes as close together as possible, so that light from the LED(s) can't directly hit the detector(s).
Are you using three different coloured LEDs and a single detector or a white LED with three detectors, each with a different colour filter?