Hi fordgtlover,
if you desire absolute silence for switching your might select a CMOS device (CD4066), an electronic switch with four units in one package. If you are just switching the input signals there should be any problem. The 4066 has an on-resistance of approx. 80 Ohms, but there is another type (forgot the type number) with lower on-resistance in the market.
Using a low noise tactile switch there is almost no noise involved switching sources.
Boncuk