I'm trying to understand the principle at the moment, and I'm curious why as soon as I install a load the voltage drops from 50-80V to 2-5V with little to no current.
Why won't the current move up instead of the voltage dropping?
In real life there will be losses, but it's always good to work out the ideal result first. If the ideal result doesn't do what you want, the real life result will always be worse, so you need a better idea.
If you ignore losses for now, this is what you get.
Unless the output voltage is very low and the off time is very short, the current in the coil will drop to zero while the transistor is off.
When the transistor turns on, the current starts increasing in the coil.
During the time that the transistor is on, the increase in current is V * Ton / L
Where V in the input voltage, Ton is the time that the transistor is on, and L is the inductance.
As the current increases linearly, the average current is 0.5 * V * Ton / L and the energy input is 0.5 * V² * Ton / L
If the switching frequency is F, the input power is 0.5 * F * V² * Ton / L
If there are no losses, that power is transferred to the output, and what you've got is a power supply that runs at constant power (if the output voltage is large enough that the current drops to zero each cycle). So at no load you get infinite output voltage. If you put a resistive load, there will be an output voltage, but halving that resistance will reduce the voltage to around 71% of what you had before. At quarter the original resistance you get half the original voltage.
So that is the best that the power supply could possibly perform without losses and it seem pretty terrible, so the basic idea needs changing. As others have said, you need feedback.
Also, I don't know what the resistance and current rating of your 80 mH coil is. If you are wanting to take 0.3 A at 12 V, that is 3.6 W, so the average input current needs to be 1 A. To get an average current of 1 A, you will need a peak current of at least 2 A and probably more.
Digikey sell one inductor that is around that. They sell one made by Hammond, rated at 100 mH and 5 A, so it's rated a little larger than you want, but not much. It would be a good starts if you want to use somewhere around 80 mH.
https://www.hammfg.com/electronics/transformers/choke/195-196
It costs over $100 and weighs 14 lbs.
Even that monster has a resistance of 0.64 Ohms so the resistive loss will be 0.64 W at 1 A, giving the best possible efficiency of 85%, and there will be other losses as well.
My take from that is that 80 mH is far too much. A lower value inductor will be much smaller and cheaper and have less resistance to get the current rating you want, but will need a high switching speed. To get anywhere near a constant voltage output you need voltage feedback.