There is no reason that magnet wire should be more expensive. From a production point of view, it's about the simplest wire for electrical use to make. It's just a single strand of copper with a thin insulation layer.
It may be more difficult to find at the hobby level though. Most magnet wire manufactured goes directly to the transformer, inductor, and motor winders. As such, the law of supply and demand for small users may run up the price.
As for "people say 24 gauge wire will work." you have to look at all of the factors driving a choice. Most people choose a wire size solely for the current that will pass through it. In your case, you need to choose it base on resistance in order to control the current through the coil. The wire is not carrying current TO the load, it IS the load.
You might try finding the right wire for your coil by a different route. Find a 5Volt, off the shelf miniature relay, and take it apart. First to see how much magnetic force it has, and second to see what wire size it is wound with. I'ts not likely to be a perfect match for your needs, but it'll give you a practical coil to use as a reference.