Here's my $0.02
Of all the comments, I tend to agree with those of JoeG and KISS. Here's are few additional comments/suggestions:
1) First, you do not say what type of plastic you are using. That can be important for several reasons. "Drilling" hard acrylic or polycarbonate can be a lot different than drilling PVC or a similar soft plastic. Second, if the hole is a little oversize, different plastics will respond quite differently to acrylic glues and space fillers, such as Loctite or super glues. I would suggest not using such acrylics, unless you are sure they are compatible with the plastic. Some safe alternatives to consider are water-based white glues and yellow carpenter glue (both basically PVA), polyurethanes like Gorilla glue, or Goop (E6000).
2) Second, drilling that large a hole with a conventional 2-lip twist drill can be asking for trouble. In many plastics, it will tend to hog in and cause cracking or a very not-round hole. You can modify the drill bit so it scrapes rather than cuts into the plastic. Grind the acute cutting edge so it meets the plastic perpendicular (sometimes called dubbing the drill).
3) Depending on how thick the plastic is, a two-lip drill will wobble quite a bit. A one-lip, step drill might give you a better result (one brand name is Unibit). You can also drill a pilot hole and then use a counterbore to get a nice hole. For a good fit to a bearing, I usually bore the hole, rather than drill the hole, but I assume you do not have a mill or similar tool for boring. I would recommend that you at least use a drill press, regardless of whether you use a standard drill, step drill, or counterbore. And, be sure the piece you are drilling is held well to the table. It is very likely to grab on you.
John