Some people use 'voltage regulator' in a broad sense. Quite often there is a source of power (voltage) that might have some variation in it and yet the load, or a portion of it, must be relatively stable, noise and ripple free. 'Voltage regulator' could be used to describe a black box that takes unstable, noisy power in and provides an output that is stable, free of ripple or noise. The stuff inside the black box can be quite different.
A resistor and a zener diode in the proper configuration can serve to regulate and incoming voltage - and OA2 regulator tube or valve can serve the same purpose. The incoming voltage must be somewhat higher than the output voltage and current is quite limited in this kind of regulator. Any voltage above the output is wasted as heat.
Transistors can be and have been configured into voltage regulator circuits- vacuum tubes can and have been as well. The circuit is designed to constantly adjust to variations in the load and the incoming supply. Input voltage must be above the output and any excess is wasted as heat. These voltage regulator circuits have been packaged into integrated circuits. The 7812 is one of them. LM317 is a popular adjustable regulator.
Another type of voltage regulator uses switching to make adjustments - SMPS (Switched Mode Power Supply) is one term. In this type of supply the incoming voltage can be much higher or lower than the desired output voltage. I suppose that in this case you might actually use the term "convertor" because the incoming voltage becomes pulsating and is then converted and smoothed to DC.
Some circuits are perfectly happy if the supply voltage moves around or has some noise in it but many circuits require a stable, noise free and ripple free supply.
I hope this give you a place to start.