How about this: if you were to put 20% down on a $500K house and finance the rest at 3.5%, you would be paying $1,167 every month in interest alone. Does that not give you the same "icky" feeling as renting?
To make it a wash, instead of buying, you could find someplace to rent for $1,167 a month and then take the extra $630 a month you would have been putting towards principle and invest it in Vanguard instead. Added bonuses for you include not having to pay property taxes and not being responsible when the washing machine explodes or the toilet leaks.
On a related note, I totally understand the appeal of having a rental property where the rent covers the mortgage (P&I, or even the property taxes and insurance). We were in that position for a couple of years. Fundamentally that thinking is flawed though. You have to consider the opportunity cost of the money you have tied up in both the down payment and the monthly cash flow servicing that rental property. Don't forget about vacancies and repairs! If you were to instead invest that money in a passive investment vehicle you may get better returns. You also may not. That is why it is critical to run the numbers first and make sure you are at least hitting the rules-of-thumb like the 1% rule that says the monthly rent should be 1% of the purchase price. I sincerely doubt you would be able to get anyone to pay $5K a month to rent a house in the Denver area, even in Cherry Creek. Houses might rent for that much in my neighborhood, but I live in Silicon Valley.