I agree with you. I look at it as an insurance policy. If something unexpected happens financially - job loss, a medical emergency, etc. at least I'm not worried about being out on the street.
This is the part I've always wondered about. Do you not have to pay taxes, insurance, and HOA or condo fees at all (this one isn't everyone)?
I'd much rather have $400,000 in cash and a matching low interest loan than nothing and a paid off house that still has fees.
To answer your first question, you have to pay property taxes and HOA fees, if applicable (and maintenance costs, etc etc - oh the joys of home ownership)
regardless of having a mortgage, so having a mortgage does not in any way save you on these costs. Sure, the mortgage company might collect your property taxes from you and put it into an escrow account, then pay the taxes on your behalf, but at the end of the day, you are still the one paying those taxes.
But let's go to your second point, because I think that's the main thrust of your position, which I understand. And it's a completely respectable thought, mathematically speaking. If I have you right, you are saying you could have a low interest mortgage, let's say 3%, while simultaneously investing your cash in the market and get returns, let's say 8%. So of course you come out ahead, right?
But then I think you have to look at, number one, human nature, and number two, sh*t happens ( unexpected job loss, illness, etc.). When I say human nature, I mean that most people will make a monthly payment dutifully, but I have less faith in people dutifully investing that same large amount ( instead of going on vacation, buying a new car, what have you). It has been my experience that most people tend towards the latter behavior, given a long enough time horizon. YMMV. And when I say stuff happens, I am referring to a persons ability to continually generate income. Of course I don't expect to wake up tomorrow and be unable to work, but it can happen. And if it did, my first thought would not be, how am I going to make my mortgage payment? Let me propose a hypothetical situation for you, by way of further illustration. Let's say your investment of $400,000 tanks in a given year ( market volatility is a given), end it just so happens that the same year, your mortgage company decides to call in the full amount of your loan. You would have thought that you could just take the money and pay off the loan, but what now? Go back to work? What if you can't? What if you don't want to?
Like I said, just some things to think about. I completely understand that mathematically speaking, what you are saying has the potential to be financially sound, but it does come with its own inherent risks.