in regards to taking extra energy from the battery, the only additional load is the relay coil, which you're paying for in either situation.
for turning back on, you nailed it ... a lead from battery positive with a switch to the positive relay coil terminal... and have a diode in-line with the poisitive lead coming from the wall-wart, so current cannot flow from the battery into the wall-wart.
I suppose a magnetic type wall wart with a linear regulator would be wasting some energy as heat, but it would be insignificant compared to any real load like a lights, television, etc. If you really wanted to optimize it, use an AC powered relay, so no energy is wasted on converting AC to DC... you would need to bypass the relay to turn the gizmo back on however. Or use a switching type wall-wart, they have a really high efficiency.