It certainly seems that the laptop wouldn't be taking enough power to flatten the battery that quickly, but it might do. You need to measure the voltage on the battery when the UPS stops working.
If the laptop has a 57 Wh battery, and the 12 V battery is 35 Ah, that is about 420 Wh, or about 8 times the laptop battery. Even if you have to charge the laptop, that should be OK.
I think that either the 12 V battery is poor, or that the laptop power supply is upsetting the UPS and causing it to overheat.
How long does the laptop last when connected to the UPS, but with the laptop battery removed?
If you wait the 15 - 20 minutes until the UPS stops working, can you still run the router and the light for a long time?
After the UPS stops working, will it work again if you leave it an hour to cool down, or do you have to recharge the battery?
What is the charger that you are using for the battery? Some are better than others.