We bought a little cabin on a nearby island for $135k (cash) five years ago. Real estate was in a slump, and we got it on a short sale, so we didn't lose any mustache hairs over the cost.
It's a nice getaway, takes about an hour to get there, with half of that being a pleasant ferry ride, so it's not a hassle to pop over on a Friday night and spend the weekend. Quick, easy access was important to us because the traffic in this area is awful.
One pain of course is the maintenance. Even though it's just a cabin, there's a small yard and a side lot to maintain with mowing, brush cutting and weeding, so we're spending a few weekends in the early summer doing that. The place needs painting soon as well. We could pay to have it all done, but well...you know...
And of course the maintenance of our primary residence doesn't stop. We have a large yard and veggie garden, which have suffered some since having two places to care for.
It is nice to have a getaway, though. My DSO has enjoyed it most, as he's miserable at work lately, and the cabin has been a good place for him to decompress.
I'm ashamed to admit this, but also I do enjoy the status of having a "vacation home". "We have a place on the island..." garners oohs and ahhs from people, and offsets their impression of me as being rather poor (millionaire-next-door-poor, unbeknownst to them). Face-punch, I know!
However, we are thinking of selling it. Beside the extra cost and maintenance, we don't use it as much as we thought we would because most things we can do over there, such as biking, kayaking, and enjoying a beach, we can do on this side as well.
Also, we're not very social so we don't have many get-togethers there. If we were more social, that'd be a good reason to put up with the extra costs and effort, because it is a cool place to hang out.
I thought about renting the cabin out to recoup some of the property taxes, but after comparing our place to other rentals in the area I realize we'd probably have to fix it up before many people would be interested. There are lots of really beautiful rentals in the area for very reasonable prices. Our place is comfortable for us as-is; a bit rustic and with second-hand furniture, and we're not too interested in throwing more money and time at it, so we won't be going that route.
Take an honest look at how much you really think you'd use it and the potential for increased value vs. the cost and maintenance issues. I agree with Widerhaken that it could be a magic place if you and family/friends use it a lot!