The biggest change is that graphics viewed in OpenOffice are now anti-aliased, meaning graphics will appear smoother, especially when you place one shape over another. It's kind of surprising that OpenOffice hasn't supported this until now. File-locking has been improved, so if you're using the suite for cross-platform document sharing you should notice fewer glitches.
Writer, the OpenOffice MS Word analog and the one I use the most, is notably changed in this version. Highlighting will now appear as a toned background gray with darker gray font, instead of white on black. On Macs, the background shows up more blue than gray; apparently this is easier to see, but I wonder if color-blind users will find the change helpful or obnoxious. There's also stronger support for comments on documents, with a new context-menu Reply option. Other Writer changes include a new grammar-checking framework to support the grammar-checking extension, and outline levels within paragraphs.
Calc, the Excel analog, includes the zoom slider that Writer was given in OpenOffice 3.0, hot hints for commonly used formulas, and double-click renaming for sheets. Chart now supports flexible axes and offers users more options for dealing with missing data points.
Although I didn't perform any benchmarking tests, version 3.1 felt like it took longer to install and appeared to hang for a few seconds randomly during use. If you notice similar conflicts or slow-downs, let me know in the comments.