Hello,
Even though you have some excellent advice here so far, I'll add my penny's worth.
The Radio Shack Lab is probably a good way to get going, especially if it includes the beginner books by Forest Mimms, which are very easy for a beginner to follow. You could, in fact, just get those books (they should be very low cost,) and the breadboard which i_build_stuff mentioned. (I would disregard everything in his second paragraph, though! You don't need to 'understand vector calculus to understand how current flows in a circuit. You can also learn a lot without the large expense of a scope.)
Don't be afraid of soldering though, it can be a relaxing and enjoyable part of this. You could start by removing components from an old radio, amp, or whatever. The best ones would be very old, before surface mount became popular. After you have some experience, then try soldering components together.
I really agree with RadioRon; a project is a great way to get going. It will give you direction. Even something simple, like a switch, resistor, led would be good for a start. You could play with the value of the resistor, and see what effect that has on your led. Learn Ohm's law by heart, then look at the data sheet for your led. Compute the current flow on paper, then wire it up and see how it works. You may even exceed the maximum current by a bit, at the cost of the led, just to see what happens.
The main thing, is to have fun. The complex stuff, like vector calculus and line integrals can come much, much later.
Best wishes,
Robert