There's no way to do it with individual letters. The way people who can read lips do it is that the whole "word" fits a pattern and they look at the facial expression to check with their previous expression and what the other person. Just try to mute a movie: You'll probably guess when a person says "maybe" by the movement of their lips, and putting the things in their context. i.e: If it is in a formal conversation, it's more likely to be "maybe" than "baby".
All this adds complexity and one has to ask himself the question: Is it needed ? Especially when you know that it's easier to acquire sound than video.