millis() is interrupt driven so delay() won't impact it, at least not on an ATmega based board.
That isn't to say that millis() is totally accurate either. Each tick of the timer is not exactly 1ms, but is 1.024ms. This error gradually accumulates until a correction is made. This can be seen in the implementation of the TIMER0_OVF (timer 0 overflow) interrupt handler.
Another source of inaccuracy is the oscillator/crystal itself, which is not exactly 16MHz. It is pretty close though, and as long as temperature doesn't change too much, is relatively stable.
The above means that you might be about 1ms out when using millis()