Very interesting topic. Here's a few insights.
Just checked the California Penal Code regarding privacy (C.P.C. was recently brought in line with Federal laws, so I think the following will be true in most U.S. jurisdictions.) CPC 632 provides that both parties to a conversation, whether that conversation is conducted in person or through some other medium, must consent to a recording in order for it to be legal. Of course, there are exceptions for law enforcement and victims of threats, extortion, etc. Therefore, he has no right to record any conversation between you and a guest. Proving that he is actually making the recording is another matter entirely.
As far as I can tell there is actually nothing illegal about pointing a camera into your window from his private property. I guest this is because until the rise of the internet the damage any individual would suffer from a video equipped peeping tom was fairly minimal. I saw a bit on the news recently describing this strange void in privacy laws. Apparently, some jurisdictions are now enacting laws to protect victims of video peeping.
Finally, (how'd this post get so long?) I'm not an expert on the technology, but I've checked the prices and it seems that the passive thermal stuff that would see through your blinds are quite expensive, so I wouldn't worry too much about that.