Gopal Adhikari:
Here is your circuit, a bit cleaned up. Please check it, and make sure I didn't screw it up. Note I labeled its interface to the outside world. I am assuming you do not have a reference, and it must be external. I am assuming that your feedback resistor divider is external.
I created a symbol for it, to make the testing easier.
Now it is simple to test to see if it regulates. The choice of the feedback divider, and the reference voltage source is arbitrary, you are welcome to change them... Here is a DC sweep of just V(in) to see what happens at V(out). You can call it an LDO, because V(in) has to be only a bit more than V(out) just as it begins "regulating". Note that with the values I chose for the feedback divider and Vref, it creates a 4.8V LDO regulator.
Now, let us learn about its transient behavior as a function of changing load: Here I switch the output load from 5mA to 50mA and back again. Uh Oh!
It goes unstable at high load, and it is marginally stable as the load decreases (because the oscillation is dying out...):
So it looks to me that you have some more work to do before we even get to the PSRR question. I tried different values of C1, and all that changes is the frequency at which it oscillates... Nothing new here, it is difficult to drive highly capacitive loads, regardless if C1 is part of the load, or if it is supposed to be a "bypass"
Hope I showed you some new insights into LTSpice...