Reading between the lines of what's written by OP, I can take a guess at what's going on now.
In standard form, the RC model works OK, but the motor and battery are getting "hot". Some heat pickup is inevitable in an electric powered RC, but I think a motor reaching, say 100 degC is normal, the RCMr's here can confirm this or correct me. The battery pack is going to get warm too. If I remember rightly, there are 'high temperature' battery packs and motors specially developed for competition RC. I'll take a good guess the OP's RC has been buillt down to a price, and is in need of improvement.
Battery pack power tools get "hot" too, but they are only used intermittently. I suspect in the RC, the motor and battery don't get much cooling. I'll guess the motor is inside a plastic housing. If it was a thermal path (metal) between the motor frame and ambient it to cool it, the motor would lose most of the heat pick up. Again, the OP knows more about the motor cooling (if any).
There's no mention by OP that the RC's motor "spins too fast" (with wheels off ground). I take that as meaning the battery cells are not too many for the motor. I'll guess it's two-cells. Is that right?
I can see why smaller batteries are tried in a bid to reduce the heat build up. I'll guess that at first when it was tried, it "solved the problem" really well, that is until the AA cells melted their way out of the battery holder!
I see the logic of trying that, and why the OP's next step was to reinstate the original battery and add a ballast resistor instead. Instead of regulating the current using the AA cell's internal resistance that is. Rheostat regulation has been tradtionally used for slot-car racing, the hand controllers are air cooled and the resistance wire can sometimes glow red hot inside.
So, my 0.02c is, the quickest easiest solution is to put the RC back to standard configuration, but add a "ballast resistor". The easiest way of doing this is do it the same way that automobile air blower fan speed control is done, a small coil of resistance wire in series with the motor, and with the coil cooled by the air flow. For resistance wire, you can ask a welder to give you a few feet of MIG welding wire, it's 0.6 mm diameter and needs crimp termination. I guess you'll need a couple of a few inches in a tight coil.