The noise you hear from the speaker might be acoustical feedback.
If the microphone can hear the speaker, the mic's output is amplified, makes an output from the speaker which is heard again by the mic and the sound goes around and around, howling or squealing.
If your earphones are completely sealed, the mic won't be able to hear them and the feedback noise won't occur.
You say the LA3220 is driving an 8 ohm speaker. It is a preamp not a power amp so has a minimum load rating of 680 ohms, not 8. It is severely overloaded by the speaker and making very bad distortion noise.
You don't describe the noise so it might be mains hum or buzz caused by not using a shielded cable for the mic.