There are many amplifier ICs made for driving 4 ohm speakers in cars and are powered by the car's "12V" battery.
A TDA7240A IC has an output at clipping of 14W into a 4 ohm speaker with a 13.8V supply. If the battery is being charged at 14.4V then the output at clipping is 15.5W. With the output clipping very badly with a 14.4V supply the output power is 20W. A pcb design is shown in its datasheet.
If you want more power then you need a higher supply voltage.
I'd like to keep it portable. I don't think these can supply as much current. Is it possible to get anything between 10 - 20 watt output with such batteries, or am i asking for too much? And if yes, how long would it last?
Your little 12V lead-acid battery is extremely heavy for its low output. It is rated at only 5Ah so it will supply 0.5A for 10 hours.
Audio is never full blast all the time (except acid rock is) so the 30W maximum from a stereo amplifier has an average of only 3W to 6W. Then the battery will last for about 10 to 20 hours. I would use a much lighter Li-Po rechargeable battery.
There is a TDA1554 car radio amplifier IC. It is a stereo amp but it gets too hot and has too many pins. It is much easier to cool and connect two TDA7240A mono amplifier ICs.