I built millions of dollars worth of spec. homes in a mountain resort setting, and sold them to folks just like you. They all have dreams of using their vacation home as family gathering place, some think they will hit it big once they list it on VRBO, and not only cover the costs, but make a good buck on short term rentals. Most think it will be a great place to escape to, many weekends a year, then become a great location to retire to, some day.
When I started, I was hooked up with an older, and absolutely ruthless shark, of a broker/realtor. She would cynically list all the fantasies that buyers had, and how it typically took five years until they showed up back at the office, and listed the place for sale, dreams squashed, bucket list checked. She even boasted of selling the same house, FIVE times, during her career. About five years into my stint as a builder, I started to notice that she was dead on. It's been almost a decade since my first project, I live in the neighborhood, and I can safely say that the majority of my original owners have moved on. Given the unhealthy regional real estate market, most of those sellers got their asses handed to them, and frequently lost at least 25-40% of their total investment, after all costs are totaled. Until you sit down and really do an honest analysis, it's hard to imaging how much money it takes to buy, repair, upgrade, furnish, insure, and pay taxes on, a second home. Particularly if, like most, you dump it in a few years, and incur all the costs of the resale. I have seen customers who took a $50K hit on a $157K place, and only used the property a dozen times in the few years they owned it. Nothing like finding out you lost $4-5K per weekend at YOUR vacation home, when you could of rented the place next door for $400 a weekend.