1. Can AVR’s programmed same way as 8051?
yes, AVR can be programmed through ISP and 89s51 can also be programmed through ISP
2. I am trying for a graphic calculator for algebra, trigonometric functions with images…which do you think will be suitable?
I am not very clear with your requirement but for mathematical functions, you can use 8051 (though it contains only add sub mul div, u can construct other forumulas)
3. If we designed some application for 8051, can it be used with AVR?
Yes
Please let me know if you need any more help.