For a parallel crystal (most common type) you add the two small capacitors to ground to stabilize the feedback loop and ensure oscillation at the correct frequency. It has to do with the feedback system and having 180 degrees inversion. The caps help do this.