It is OK for the power supply to be able to deliver more current than you need. As long as the voltage is right, your load device will just draw the current that it needs.
Other than it's internal operating power and inefficiency, the converter will only draw the input current necessary to supply the output current.
But, one thing that you need to understand. Since the output voltage is 5 times the input voltage, the input current will also be at least 5 times the output current.
Input power will always be greater than the output power.
......so 10mA output at 24V = 240mW of power. That means for an ideal 100% efficient 9V-to-24V converter you would need 240mW / 9V = 26.7mA battery current. In practice the efficieny might be, say, 85% and the battery current would then be ~ 31.4mA.