Yes, it's quite simple. Tandy/Radioshack used to have a kit for it which used cheapy plastic fibre just like the one you've described, not that I ever pushed it to 100MBps.
The best I ever did was send some video over it.
Simple OOK is all that is required to make it work. I used a simple high impedance gate using a 4069 to make the IR transistor drive the levels suitable (enough) for RS232.