your regulator will fall out of compliance if the load is too small.
Also, your load had nothing to do with your virtual ground, why do you need the virtual ground?
also, an op amp works quite well, why stack regulators?
A virtual ground circuit frequently has a sourcing and sinking ground current. Your lower LM317 can source lots of current but sinks no current.
Your circuit is missing the recommended input and output capacitors shown on the datasheet of the LM317 so it will probably oscillate.
Here's a split-supply circuit that uses two voltage regulators that can either sink or source current, so can handle any load current imbalance between the two outputs..