Hero,
That is exactly true, and current drive is something that has been experimented with for a long time. It totally nullifies any back EMF induced or Le induced distortion. Unfortunately, due to the theoretical infinite output impedance, you have no electrical damping (in Thiele/Small parameter terms, the overall system Q is determined solely by Qms). This means that the speaker will resonate severely at its natural frequency (say 50Hz for a 6.5" speaker typically). Typical damping values for electrical Q are 0.2 - 0.5 for quality speakers, while mechanical Q values are 2-10 for the same quality speakers (Q being the inverse of damping). This means that in a voltage drive system Qes dominates (total Q is Qes*Qms / (Qes + Qms)) and in current drive systems Qms dominates.