This vague comment...
Some financial decisions we make are basically idiot proof. E.g., if you save for retirement using tax-deferred accounts (like an IRA or a 401(k) plan) and use cheap index funds, you basically can't lose. And you basically can't make a mistake.
Your decision IMHO isn't like that. You can make a great, good, fair, so-so or terrible decision.
I'd say that some of the comments you get here and elsewhere are just reflecting this.
But I also think if you're super-careful you can get something like what you propose to work great.
I said this in another thread active right now, but I'd try to turn your cost of using into a nightly rate and compare that to what you would pay for nightly lodging if you didn't own the property.
A handful of notes:
1. If an honest denominator gets large, you can make the economics work. E.g., if place costs $16,000 (which is probably low) and you use 40 nights a year (which is a LOT), your nightly cost is $400 a night (which is "Ouch!")
2. If the place costs $12,000 and you use 120 nights a year, your nightly cost is now $100... and that might be reasonable.
3. If you can truly cover a ton of your costs via rental income, that'll make a big difference. E.g., if after all rental income and expenses, your true cost is $1000 a year and you can enjoyably use the property for 100 nights, you're use now costs $10 a night. That would be a good deal for many people.