The output is TTL compatible. It is not 74HC compatible. You need to use a 74HCT frequency divider.
This part is a phase locked loop (PLL). If you don't have experience with PLLs, I'm not surprised that you are having difficulty. You are probably better off leaving out the frequency divider to start with. As the divider ratio N goes up, the loop gain goes down, and lock is more difficult to obtain. I'm not sure how high N can be and still have the part lock.
Do you know that, if you set the VCO free-running frequency to F, and the divider ratio to N, that the input frequency must be F/N? What are you using as a signal source? And why do you want to use a PLL to generate your pulses?