And of course a more efficient way would be PWM.
Problem with a potentiometer is power wasted and heat. If you decide to use a potentiometer, please consider a power resistor in series with the pot and motor of a low enough value as to not deter from the circuit but to current limit the motor. It's a small motor but .25 watts isn't much for your pot and it might get very hot otherwise.
Look into PMW, it is way more efficient and friendlier to your components.