C4 is nowhere near big enough. Re-run your sim with 1000uF in place of the 1uF. What you are seeing is the LM317's feeble attempt at recovering to a sudden increase/decrease in its load.
The root cause of your problem is the 47uF capacitor at the output that must be charged in nearly zero time...
To turn this into a practical circuit, make the 47uF capacitor like 0.47uF, and follow that with a voltage follower opamp. Then the bypass on the LM317 can be a "normal" size, like 4.7uF.