You would need a boost regulator module rated at 5V and probably 2 - 3A out, depending which Pi version it is.
The input range should be suitable for the lipo cell range, eg. 4.2 down to 3.3V
A module intended for a DIY high current power bank may be suitable? That should also include the charge circuitry.
Connect to the 5V at the expansion connector or under the USB socket on the Pi where it draws power, wherever is convenient.
I've used a chopped up USB cable to run a Pi in one of my 3D printers, just so it's easy to swap to another PSU if needed for testing or setting up.