I'm not sure what other things you should delete, you could try a deltree from dos on the temporary internet files folder, maybe the history folder as well. But I've always found that Windows 98 installs really start to go sour after a while. It doesn't clean up after itself very well, and becomes less speedy and stable, even if you defrag it.
I personally would go for a reformat, it's always nice to have a fresh install of windows that you know for sure doesn't have any spyware, viruses, or superfluous programs running in the background. And if your finances permit, I'd also get either win2k or xp, I prefer those to win98, and consider them more stable.