Okay, finally succesful run. I made some measurements, got weird ass numbers, so figured I did something wrong. Did some more research...
Instead of driving the LEDs from the output pin of the LM317, I need to come off the Adjust pin (wrong side of the resistor). Also learned that I needed 3 volts above what the LEDS needed, so for the 5.4v volts from the NiCds, only had enough for 1 LED. Left it running at 7:30pm, it's still bright at 1:30 AM.
One odd note, measured 64.8 mA with the meter between the LED and neg. battery. The resistor is 10 ohms.