Answering your question, an optical tach sensor and a piece of reflective tape is everything you need. See the picture below. This sensor will provide a TTL pulse string corresponding to rotor position.
For "real world" size machinery, you can also use an inductive sensor if the shaft has a keyway. Again, this sensor provides a once/rev TTL pulse stream. Picture below.
However, there is a basic flaw in your reasoning. If you are trying to counter mass imbalance, an out of phase force equal to the imbalance could counter it.* But you
will not be able to generate this force with an eccentric mass on a motor. The effect of the eccentric mass varies with speed. But you have a fixed speed to counter the force of imbalance. You have no means to vary the force produced by your motor.
* An imbalance typically generates a radial force at 1x rotation rate for a shaft supported on both sides of the rotor. An overhung rotor, meaning it's supported by bearings on only one side of the shaft, will generate radial and/or axial 1x force depending on the exact geometry. A small-scale model will not accurately reflect the real-world situation.
View attachment 109876 View attachment 109877