I seem to have opened up a can of worms here.
Having read my old college textbooks on the subject, it is possible to get very confused, it all depends on what is happening to the inductor at the time, if energy is going in or coming out.
In figure 1, the battery is making a current I flow through the inductor, a magnetic field is generated and stores energy.
If we measure the voltage across the inductor, we see a +ve voltage (measuring from the reference point Ref).
If we now close the switch, (figure 2), the battery no longer drives current through the inductor. The magnetic field collapses, putting energy back into the circuit, trying to maintain the current I. To maintain this current flowing in the same direction the voltage from the inductor is now in the opposite direction from before. On our meter (or oscilloscope) we will see a -ve voltage across the inductor with respect to the reference point.