In reality, the range of any radio transmitter isn't so much dependant on modulation type as much as frequency. There are optimal frequency bands between 160 meters and 6 meters which can transmit a great distance given ionospheric conditions and sunspots.
FM is typically not used in lower frequency radio because of the bandwidth requirements and the available "real estate".
In your case, it sounds like you want to use the accepted "commercial" frequencies for FM radio, for which recievers are commonplace. This would typically limit transmissions to Line-of-site and a measure beyond depending on terrain, amplifier power, and antenna height,construction. You would be amazed at how much you can do with very little power. I have QSL cards using CW (morse code) communication from clear around the world using less than 1 watt.
A few suggestions that might "help", assuming you can construct a maximized transmitter...
Antenna height. If you are in a mountainous area, that tiny transmitter might surprise you at elevation. If you are doing this for fun and not a permenant set-up, you may think of a portable radio station on a mountaintop. If you have the ability to weather-proof your equipment and have sufficient battery power, and a secret location, you might have fun doing it that way with an automated transmitter.
ANTENNA. You mention 5-miles *around*, and this would be a typical omnidirectional vertical antenna, but a resonant wire dipole will give you bidirectional coverage...if you aim at population centers you will achieve some gain (which is a misnomer in a way, but in other words you concentrate the transmission and reduce loss)
LOSS...important concept. Transmission lines intoduce loss with standing waves and resistance. The better your transmission line and all of your connections, and the shorter the line, the more use of your transmitting power that will not be wasted as heat.
If you have the inclination, a YAGI style antenna (think outdoor tv antenna) has maximized gain in a single direction. The reflector dipole elements help focus transmitter power. Multi-element yagis further do this. It is the equivelant of mirrors and lenses in an optical system, only with radio waves.
The antenna needs to be a full wavelength above ground, more if you can. Given a 10 meter antenna as an example, it will work optimally at 30+ feet above the ground.
Also, some evidence shows transmitting across saltwater helps with gain, though I doubt this will apply in what you are doing.
Some or none of this info might help you, but perhaps you did learn something new or someone else.
Oh one more thing...polarization. (Most) Any antenna style can be polarized vertically or horizontally, and with short range transmitters (for which even 5 mile radius includes) it is important that the recieving station be polarized the same way, It has been a while since I played with my radios and built antennas, but 10-100 dB of loss occurs with cross-polarization.
It might be educational and fun for you to refer to the radio amateur's antenna handbook and googling "radio antenna", "dipole", "yagi", etc to get construction ideas and formulas for your frequency specifics.
Remember, a resonant antenna is by definition a function of radio frequency. A dipole is 1/2 wavelength, with 1/4 for each element to be efficient. There are off-center and multi-wavelength designs as well that give differing patterns for the transmission, but it is very hard to beat the standard 1/2 wave dipole.
And for transmission line, if you have the ability to match the impedance of your transmitters amplifier properly, parallel ladder line is very good, low loss stuff and you can construct it yourself with copper wire and popcycle sticks.
It is IMPERATIVE though, for the sake of your transmitter amplifer, that the impedance of your antenna is matched to the output impedance of the power transistor. Matching networks can be built, and that is, unfortunatley, where RLC circuits are neccessary.
However, matching can be done with the dimensions of the ladderline as well, along with methods you can employ found all over the internet.
Bad consideration is that many of these projects require at a minimum an SWR meter or an antenna analyzer to tune them to resonance.