Welcome to ETO, Ramanta!
Recheck your wiring with the pots in place. Since both channels work with a direct connect, a mis-wired pot is the most likely problem.
You also forgot to insert the 0.1 or 0.22uF caps (try them both, separately, for what you consider the "best" sound) in series with the signal from the pot(s) to the input of the amps (as per the instructions you included). The caps are necessary for isolating the iPOD circuitry from the amp inputs (among other issues).