Other than the viewfinder size, the only other annoyance I can think of is the lack of full manual exposure control when shooting video...
the video quality was more than sufficient for my basic needs...
At the moment, despite the above-mentioned shortcomings, the D5100 is the best entry-level Nikon DSLR on the market. I would not hesitate to buy it even if you are able to find a D90 at the same price point. Is the D5100 worth upgrading to? If you have a very old entry-level or semi-pro DSLR like Nikon D40/D60/D70/D80, then it certainly is. However, if you have a more recent camera like the Nikon D5000 or D90, I would skip at least another generation DSLR before considering upgrading and invest in good lenses instead.