The general rule for car trading questions is keep the one you've got unless it's a total clownmobile. The Outback is a moderately sensible (though not optimized) car for your needs, so keep it.
There are switching costs associated with getting a different car. You'd have to save a lot on your commute to overcome them.
Figure you're going to work 260 times a year and driving 50 miles each time. If you are in a Prius getting 50 mpg, you need 260 gallons of gas each year. If you're doing it in your Outback getting 28, you need 478 gallons. With a Prius and gas at $2.50 a gallon, you would save $550 a year on the extra 218 gallons you'd no longer have to purchase. Yeah, that's savings, but it might not be real savings if you have to spend a lot initially to start harvesting that savings.
As everyone's pointing out, trading up in cars is going to cost you some decent sales tax and registration fees. Plus I doubt you can find a Prius for a similar price than your Outback can fetch, so there goes a couple thousand more. The payback period would probably be at least 5 years...but feel free to run your own figures and decide if it's a good choice for you.