Wow, this blew up a bit while I wasn't paying attention!
Okay, so, let me reply to a few of these...
Are you maxing out your contributions to your tax-advantaged accounts such as 401K? This is $18k for this year. At your income levels and tax bracket (I guess 25%+ including state taxes), this can reduce your taxes significantly. If you get married, that will be a total of 36K. This tax advantage space is on a use-it or lose-it basis every year. I would first do this before prepaying the mortgage.
Yep, I'm maxing out the 401k and the traditional IRA already. I'm planning to try the Roth ladder strategy from MadFientist later in life.
To answer your question: No, I would not pay the mortgage off ASAP. Diversify. You're pretty young. You might consider a 15 year for a better rate and a built in quicker payoff.
For maximum mustachianism, you might consider a lower priced house. While you can pretty easily afford the house with the 400K mortgage (2.7X income), in the long run, you would be better off with maybe 2X income for a mortgage. With that will come lower taxes and less maintenance and more money to invest making you wealthier.
I asked about a 15-year loan instead of a 30-year one, and according to the loan officer, there would be a 0.01% cut in the interest rate. He didn't seem to think it was worth the higher (minimum) payments, but he did offer to work up the numbers for me if I really wanted to see how it would shake out. I decided I'd rather have a lower minimum payment with the freedom to choose to add as much as I want each month.
We've looked at a bunch of houses in a variety of price ranges. Maybe the start of the year wasn't a great time to look for a new house, but things have gotten pretty crazy in the metro-Denver area, and at least a half-dozen times since the beginning of the year, houses that we were interested in that were in a lower price range went under contract literally hours after they were listed, so we didn't even have a chance to look at them. If we could have found a house with solid fundamentals (modern-enough wiring, no cracks in the foundation, close enough to bike to groceries etc) for less that didn't disappear before we even got to see it, we would have jumped on it. When it comes down to it, the mortgage is definitely more than I would
want to pay, but it's definitely something I
can pay, and I can even afford to pay double or more if I don't put all of the extra money into investments. So I'm not
too worried about it.
You are getting ripped of, that interest rate is way too high. Penfed is 3.625% for a 30 year fixed
Unfortunately, my credit is in the 650-ish range because of some poor choices earlier in my life which will hopefully fall off the credit report in the next two years. I have friends with credit scores in the 750+ range who just bought a house in the SF Bay Area, and they got a 4.0% interest rate, so I was actually pretty happy to get 4.375%. Besides, even if I don't pay it off as fast as I possibly can, making some extra payments (especially the first year) should cut the total interest paid by a pretty substantial amount.
This wasn't mentioned earlier, but make sure that your mortgage does not include any early payoff penalties that would negate the benefits of early payoff. Even if you don't pay down early, leaving the option open is a good call.
Already confirmed this with the loan officer at my real estate agent's office. No early payment penalties! He claimed he hadn't actually seen a loan with early payment penalties in the past several years.
At that interest rate I would pay it off ASAP, and in your shoes I'd either buy less house or save more to put down on it. I'm risk and debt averse, value stability and not being tied to a job I'm less than satisfied with. I am aggressively paying off student loan debt and once that's done it'll all be funneled into the mortgage before I build the stash beyond a 401k for employer match. If I had no debt and lost my job I could get by on side gigs or whatever came along indefinitely in a worst case scenario. Your values may differ, everyone is different.
I've got more saved (an additional $30k in liquid cash at the moment beyond the down payment itself), but I don't want to put ALL of my savings into the down payment. I like having the safety net. And of course the loan officer is trying to get me the biggest loan possible, but I'm still going to be putting down just over 20% of the house's cost as a down payment. I'm not really very risk averse, but I also value stability, and I
love my job. Even once I hit the FIRE savings point, I don't plan to quit; I'd rather cut my hours down to 60% and take the 40% pay cut than quit the job, because I enjoy it so much. (Of course, that could easily change over the next decade.)