There are two reasons to filter an A/D input: reduce noise above the input frequencies of interest, and anti-alias.
Generally you want a filter to roll off the signal above the highest input frequency of interest to suppress noise above that frequency, which would otherwise add undesirable noise to the digitized signal.
At a minimum you want an anti-alias filter to suppress the input above 1/2 the A/D sample frequency, even if there are signal components above that. Any noise or signal frequencies above 1/2 the sample frequency are digitized and appear as low frequency (aliased) components in the digitized signal. This adds noise and difference signals to the digitized signal. For example, an input signal exactly equal to the sample frequency will appear as a dc shift in the digitized signal since the sample will always occur at the same point on the waveform.
If you took a 10KHz bandwidth music signal and fed it to a 10k sample/sec. A/D converter with no anti-alias filter, the digitized output would sound terrible when converted back to analog and played through a speaker. This is because all frequencies between 5kHz and 10kHz have been aliased back into the 0 to 5KHz band. (Nyquist's sample theory says the sample rate must be a least twice the highest frequency of interest. Thus a 10kHz sample rate can reproduce a maximum of 5KHz.)