The crystal looks inductive in a narrow region near its resonance (actually, between its series and parallel resonances). That's 90 degrees of phase shift. You need another 90 degrees (provided by either of the caps) to get 180 degrees phase shift between in and out, which is the main condition for oscillation. That's about as simple as I can describe it.
Not the capacitors are a requirement and aren't optional, although if you've got large PCB traces it is possible for it to oscillate without capacitors but it's a silly idea and not stable.