Well, without seeing your exact circuit and schematic it is difficult to pinpoint what might be happening, but certainly there shouldn't be any difference between using alligator clips vs. using the 9 volt battery clip.
It doesn't sound like a ground loop, sounds (possibly) like you have some kind of inductive or capacitive coupling issue between your leads that is minimized by the 9 volt clip. Perhaps you are setting up an oscillation that is being amplified. Just a note from the datasheet, and this is for the AM amplifier application so I don't know exactly what you are doing, but it mentions twisting the power and ground leads. You might even use shielded twisted pair wire for that purpose. Worth a shot.