At line level, the single relay will work fine. Using electret (amplified) mics, it should work fine. Sorry to burst your bubble, but the inputs to computer sound cards are single ended, meaning that the sleeve of the input jack is just tied to the computer ground, and this is an unbalanced input; there is no need to switch both sides of the audio line.
I have built interfaces to Computer Soundcards, and the biggest issue you are likely to encounter are ground-loops between your external equipment and the computer ground (the switching power supply inside the computer is the culprit). I had to use audio isolation transformers between the external equipment and the computer both in the Line-IN and Line-OUT connections to get rid of the ground loops. Hams make extensive use of Sound card INPUT/OUTPUTS to interface their HF radios for various digital communications, and audio-coupling transformers are essential. Google "RigBlaster" for an example of a commercial product which addresses this issue. Using an isolation transformer makes for a balanced input, where neither side of the line is grounded.