5v from a single Lithium polymer battery is easier than 3.3V because a LiPo generally runs from 4.2V (at full charge) down to ~3.1V (considered discharged) so the battery voltage is always lower than the converters output (5V).
At 3.3V the converter must either have a boost and an LDO (these exist) or a buck-boost (more common) because the input can go above/below the output.
Although everything runs from 3.3V, I would go for a lipo-5V boost module. This may not be as efficient (power loss in regulators, converter loss, parts draw more power at 5V than 3.3V etc..) it will be more convenient as boards that both boost lithium polymer to 5V, and have a charging circuit for said battery are quite common:
https://www.sparkfun.com/products/11231
That is apparently a 'buck/boost' so can supply 3.3V or 5V from a lipo cell.
Lithium batteries are handy because they can have high capacity for their size, fairly ubiquitous, and for 'handheld' devices that tend to be flat, the cells themselves can be as thin as 4mm. The downside is they require specific charging circuitry, often from USB for cells with <1000mAh capacity. But with sparkfun/seedstudio/adafruit industries knocking out small charger modules for a few $$ it has become a mainstay for hobbyists to power their projects. If you're planning on going into production, of course it would be better to design your own power supply, to drive down costs and add facilities such as - powering device from charger input while the battery is charging. But for proof-of concepts, and one-off's it works just fine.
Beware of boost converters, often (but no always) when 'disabled' the output is connected to the input via a diode. This means if you disable the 5v converter to turn your device off, the battery will still power it, just at 3.7 - ~0.4V = 3.3V. Adding a switch in series with the battery works, but then you have to make sure that is 'on' when you charge it
I think the sparkfun device above has 'true disconnect', so you can power off your device from your microcontroller.