Sorry - I should not really have called it mixing. It is simply that one is on while the other is off, and I don't want to blow up the output of the other.
I'm using a LDO schottky diode, and I've found that my FET does turn on at the 2.9v there is on the gate. It is a small speaker, so even thought the Rds of the FET is ~1.5 ohms, that's no problem in this case.
Andrew