Suppose that you have preamps each with a gain of 500. But their noise level is 1/500th so the output of the 2nd one is overloaded with noise!
1) You can measure the gain of a preamp with an AC voltmeter if it is calibrated for the frequency you are using. "Ordinary" multimeters measure 50Hz or 60Hz mains frequencies accurately but are very poor at other frequencies. The AC voltmeter also must have a scale that is sensitive enough to measure the low input level and a scale for the high output level.
2) You should also have a 'scope connected to the output so you can see if noise or clipping is ruining the measurement.
3) The preamp should be driving its rated load resistance.
4) Duh! Then the gain of the preamp is simply calculated as the output level divided by the input level.
Of course to the input of the preamplifier you must connect signal generator which signal must correspond to the preamplifier nominal level and frequency.
The input to the preamp will no doubt be too small to measure. Set the signal generator output to a level than can be measured, then feed it thru a potentiometer to attenuate the signal to the preamp. Adjust the pot such that the preamp output is equal to the signal generator, then measure the pot resistance to calculate the preamp gain.