One mortgage here (but second time I've bought a house). Putting 20% down usually avoids mortgage insurance and thus a cost saver in the long run. I have a hard time seeing any advantage from putting 50% down unless you've maxed out your retirement accounts, your kids' college saving accounts, minimized your recurring expenses, and paid down any outstanding consumer debt. Say you haven't done the above, but do go ahead and put 50% down. Paying off the house quickly will protect you from foreclosure/eviction if you lose your job in 5 years. If you are concerned about job loss resulting in homelessness, I think then it's a valid point. However, there is the opportunity cost from not investing, and those potential interest earnings would be helpful if you are unemployed in the future.
If, in several years, you have enough to support the family after retiring, and have done the above to maximize your tax deductions, then go ahead. You'll lose some several thousands of dollars over the long term if your investments do well, but that probably won't affect your lifestyle. If your investments do poorly, then you may save a small amount of money that otherwise would've been lost & it again won't affect your lifestyle.
Thus, if you are in a precarious job situation and want to secure housing for your family in the long run, consider paying the mortgage quickly. However, most people who would benefit from this scenario probably don't have the money to do so, and conversely most people who could do so probably will have minimal benefit from it.
I'm about 10 years behind you in age and family situation. My plan is to get a 30 year mortgage to keep payments low, maximize tax-advantaged accounts, and put the rest of the savings in taxable accounts (all in index funds). If the kids are set up for their college, and we have sufficient savings to retire, then we'll probably pay the mortgage to get it off our books. This is with the understanding that it'll cost me some (hopefully trivial) amount of money over the next several years.