CPU monitors do not tell you the whole story of what is going on deep in the system. In fact, one of the biggest performance killers is virtual memory. Even with a small load on the CPU, over-use of VM can bring the fastest of machines to it's knees.
Buying more RAM may open up the bottle neck that is killing the performance of your computer. VMometer will help you determine if your computer would benefit from buying more RAM.