Have you looked at the output on a scope?
It's a switching regulator so there will be a high frequency ripple on the output, you need to filter the output, first try a 100nf ceramic capacitor on the output (electrolytics are no good at high frequencies), failing that you need to build a propper filter to remove the ripple.
Failing this, adjust the output of the switcher to a slightly higher voltage, (like 4V) then add a 3.8V ultra low dropout regulator on the output, this configuration is known as a hybrid regulator.