Wikipedia give a good working definition
Instead of having several different instructions to load and store data to various registers or memory an Orthogonal computer like the PDP11 use a MOVE instruction. This one instruction can replace all CICS or RISC instructions that move data.
The MOVE instruction has source and destination fields that can specify any resource: register, memory, PC, STACK. You can move anything to anything in one instruction. The same is true for all of the small set of instructions.
I think the reason we do not see more of these machines is that it is complex to create the data paths needed to make all the addressing modes work. And given that little production work is done in ASM the beauty would be hidden anyway.
Once you have programed ASM on a Orthogonal machines others seem to be a painful set of special cases. Which they are.
Digital equipment was on the right track with their LSI-11 series of microcomputers. They used a microcoded processor to create the instruction set.