As long as you are only replacing the battery, you do not need any changes to the rest of the lamp or to add any resistors etc.
Normal "Dry cells" start at 1.5V each but the voltage gradually drops as they discharge, ending at about 1V each when totally used up.
That means a device intended to work on three series-connected dry cells should work OK on anything from 3V to 4.5V and the voltage of a single lithium cell is well within that range.
When paralleling lithium cells they should all be identical and at the same state of charge (the same voltage) when you connect them, to avoid high currents between them and possible damage.
As alec_t say, you must still use a properly made Lithium cell charger to recharge the battery pack.
Any overcharge of a lithium cell can have catastrophic results; it's common to add an extra "protection board" between the cells and whatever they are connected to, as a belt-and-braces type safety precaution in case of a charger fault.
They also protect from over-discharge, which can shorten the life of the cells.
There are lots of places selling protection boards on ebay. Just make sure the current rating is somewhat above whatever your setup will ever use for charge or discharge.
eg. **broken link removed**
Some cheap devices rely totally on something like that to limit charging, but it's an invitation to disaster; you must still use a proper charger for safety and long cell life.