If it were me and at one time I had access to a mill with readout but not CNC. I see the USB port to be the most problematic.
I'd use pilot point bits or step-drills (Unibits) for round holes. Hole saws are another choice and for odd dimensioned round holes a sort of fly cutter. I have a Dewalt set of pilot point bits and also a three set Unibit set at home now.
At home and what I used to use at work, I have something like this:
https://www.boschtools.com/Products/Tools/Pages/BoschProductDetail.aspx?pid=JS470EB Variable speed,, T shank, and supported by a roller. The jig saw also supports various amounts of "wiggle". It moves the blade away from the work to clear the chips. Great for aluminum.
Take some double-sided tape and put it on the bottom plate, but DO NOT remove the other side. It will provide a mar-free sliding surface. Drill 4 holes at the corners for the bit and file square.
Your other choice is a nibbler.
Options for the USB hole would be a jewelers or coping saw and jewelers files. Drill the hole at the ends and insert the blade through the hole. If you had a hardened lathe tool blank, this could be used to outline the edge when gripped in a vise and your filing and/or cutting.
Normal drills cut a triagular like hole. The step-drill and the pilot point/brad point drill bit won't.