There are a few problems with this plan.
#1: You will need to be neurotically clean (perhaps just tie the kids up in the closet when they're home?) and this is going to cramp your family style. Trust me. I know how it works. We still use our couches from grad school because hey, what's one more vomit stain or melted crayon at this point? Aw, sweetheart, it's ok - mommy barfed there too after her orals afterparty!
#2: Being an absentee landlord sucks even with long term tenants. With short term renters, you do NOT want to be out of town unless you are going to hire a professional manager ($$$ not going in your pocket). People will need help with the simplest things, the WiFi router will need a reset, the drain will clog, etc, etc, etc. If you're around to handle it, it's dead easy. If you're not .... angry guest, bad reviews, no bueno.
#3: There are legal issues. Your city/county may prohibit short term rentals, your HOA may prohibit them as well. Even if totally legal, your neighbors might hate the idea and you could sour relationships with them.
#4: If you really need to rent to high-end clients, you need nice stuff in your place and you need to have the place rented all the time. Part time whenever you feel like going on vacay is not going to cut it to cover your costs to buy fancy furniture (and keep in looking nice).
My advice is that you look into one of the home-swapping sites - someone who lives somewhere really neat (and also has kids) might want to come visit your location, while you visit theirs. Or build some sort of separate space like we did.
-W