NiMH cells self-discharge too quickly to be used in something like a remote. Personally, I do not consider a mouse to be a low load. Such low loads do cause deteroriation (not necessarily damage) to lead-acid cells. Not sure about NiMH because no one uses them for that due to their self-discharge.
Don't know about the charging thing though. Try a better charger? NiMh dies out after 200-300 charges. If you fast charge them I think it cuts their life by 1/4 to 1/3. You probably wouldn't notice it.
Voltmeter a cell, see if it's polarity is reversed and see if it's below 10V (both are bad, duh).