P3 is, by design, a very sensitive control and there may not be anything malfunctioning just because it seems that way. If you want to make it easy to manage, you could use a multi-turn trimmer pot, or you could break P3 up into two fixed resistors and one variable, to reduce the range of voltage adjusted by P3.
Measuring the output power is most easily done by checking its range as you are doing. However, I think there is more going on here than you are telling us. The schematic shows a direct connection from the antenna to a capacitor, yet you said that you have coax between the two. This changes things a fair bit. Can you explain or show by picture or diagram, exactly what lies between the output transistor and the air. Closeup photos of your connections, your circuit board, your cables and your antenna would be useful.