Pay off 3.38% mortgage on reduced income? (not FIRE yet)

[Mako52]

I have been thinking about paying off our mortgage and would appreciate insight from those who have or have been on the fence.  Plenty of forums talk about the “peace of mind” of not having a mortgage.  We have solid net worth but I am dealing with 18-24 month reduced cash flow due to industry and client changes.

Our goal is to maximize our long-term net worth, minimize current SWR, and retire/move/downsize in 9 years.  Keeping ARM mortgage helps cash flow on reduced income but decreases home equity at time we plan to move. 

Our situation:
Married couple, Mid 40s, 22% Federal bracket, 5.75% state income tax, $9k property tax, itemized deductions $45k.  $1M in land we own and rent out yields 40k (included in gross income numbers below)

Home value 860k, mortgage balance 452k
Roth IRAs 121k
Taxable Retirement  830k
Brokerage, CDs, etc  980k
529 Plans for 2 high schoolers 131k.....plan to send them to in-state 4yr public school
Health Savings Acct 19k
Refinanced from 4.13% to 3.38% 10/1 ARM earlier this year.  Reduced P&I from 2800 to 2000.
Expected withdrawal rate next 12 months: 2 to 3% of portfolio

Pretax Income levels required to not withdraw from portfolio.  (State and Federal Income tax factored in)

Mortgage Options:
Keep ARM: $170k gross income needed.  We will pay 15k Federal, 7k state, 9k property tax. 
Recast ARM?  2k P&I isn't "painful" for us. 
Refi to 15 year: $192k gross income needed.  Stay in 22% tax bracket. 
Pay down 150k on mortgage balance, go with 15 yr at 3%.....$170k gross income needed

Pay off mortgage entirely: $143k gross income needed.  We will pay 14k Federal, 6.5k state, 9k property tax, still be in 22% bracket

Monthly P&I is currently 20% of our expenses, not including Federal and state taxes, FICA, 401k contributions, etc.  If you factor Federal, state, and FICA taxes it’s around 14%.   So 80+% of our other expenses won’t go away if we pay it off.   

Over the last 7 years we threw $48k in extra principal payments towards the previous 4.13% mortgage. 

While we’ve enjoyed the last 10 years of the stock market, as everyone has, we also experienced 2000 to 2010 and saw atrocious stock ROI. 

Pros to mortgage payoff:
•   Reduces annual gross income requirement by $27k.
•   Less to stress/worry about. 
•   Saves $120k in interest through our target move/downsize date, increases home equity by $360k at expected move date. 

Cons to mortgage payoff:
•   Ties up a significant chunk of our nest egg in an asset that is barely appreciating.  The home’s value has gone up maybe 15% in the last 10 years.  If we pay it off, nearly half of our net worth will be in real estate, until we sell this home and downsize. 
•   Will we fret over having that much illiquid capital tied up? 

Between keeping the ARM, paying the balance down, recasting, or paying it off entirely, which option would you go for?

For those that chose to pay it off while still working, did you already have significant equity in the home?  What % of your monthly expenses was your mortgage? 

[RWD]

Why are you talking about requiring a certain level of income? You've got $2 million not tied up in real estate, that's plenty to pull on to make any and all payments even if your income is $0.

3.38% is an excellent rate and you should keep those terms as long as you can by making minimum payments.

Investment Order
Club

[Boofinator]

In your situation, I would keep the ARM. It has a good interest rate, and with a 10/1 it sounds like it is unlikely to increase before you move. Additionally, when you move/retire, if you don't have all your money tied into equity, you'd have more than enough to buy your new place in cash without having to worry abut selling your old place first.

Recast might be a good option, if the associated fees are negligible.

Yes, stocks certainly come with risk, but over 9 years the chance of losing out to 3.38% (less itemized deduction tax, so ~2.5%?) is a bet you should be willing to make if your objective is to maximize net worth within that timeframe.

[talltexan]

I invite you to come over to the Don't Pay Off Your Mortgage club and celebrate that excellent long-term mortgage rate with people who are maximizing their net worth!

[Mako52]

Lender (Loan Depot) has offered us 3.25% on a 20-year fixed with really minimal closing costs, but we need to lock in ASAP.  We are barely into the 30 year amortization schedule on our 3.375% ARM, and I'd really prefer to keep the flexibility of not dumping more $ into a house. 

What would you do? 

[talltexan]

I don't think you need to rush like it sounds like they're making you do. Bond market will stay low for quite a while.

Of course, I've been wrong before.

[Laura33]

Lender (Loan Depot) has offered us 3.25% on a 20-year fixed with really minimal closing costs, but we need to lock in ASAP.  We are barely into the 30 year amortization schedule on our 3.375% ARM, and I'd really prefer to keep the flexibility of not dumping more $ into a house. 

What would you do?

Keep the current one.  Your current concern is cash-flow.  So don't increase your monthly payment by moving to a 20-year in the middle of a cash crunch -- and certainly not for a 0.15% change in rate. 

Generally speaking, it is good to have a paid-off mortgage when you retire to lower required monthly outlay and so minimize SORR; but you will do much, much better in the long run if you keep your money invested until then. 

The bigger issue is what are the expenses that require you to bring in $143K gross without a mortgage, just to cover your other mandatory expenses?  That's the 800-lb gorilla in the room.

[Boofinator]

Lender (Loan Depot) has offered us 3.25% on a 20-year fixed with really minimal closing costs, but we need to lock in ASAP.  We are barely into the 30 year amortization schedule on our 3.375% ARM, and I'd really prefer to keep the flexibility of not dumping more $ into a house. 

What would you do?

Could you elaborate on "really minimal closing costs"?

[Buffaloski Boris]

I guess I’m going to earn my contrarian chops once again.

I agree with Laura above. One gorilla in the room is the $143000 in other expenses. I’m going to make a few assumptions here: (1) you live in Northern Virginia, (2) your kids will attend public universities in Virginia and you will pay for it, (3) you’ll relocate to a lower cost area at some point.

If my assumptions are correct, then my answer is none of the above. The second gorilla is college. The best investment I see in this scenario is the VA 529 plan. Public universities in VA are overpriced and parents will spend a pretty penny to send your kids. Look at expected costs to attend UVA. Pucker time!  Contributing 4K per parent per kid (16k total) gets you an instant 5.75% return as a state tax deduction. Sweet.

After that if I’m in this scenario I’m plowing as much as I can into the mortgage. It’s a 3.38% return on what’s essentially a no risk investment. I cant get that from bank accounts or t bills.   But wait, there’s more! While I don’t think you’ll get much in the way of financial aid, you might get some. And equity in your house is not counted at all as an expected contribution. Nor are retirement assets. You need to optimize there and I think it’s a 2 year look back on parental assets.

[Mako52]

@Buffalo Chip - Here's what the VA 529 page says about tax deductibility:

"Contributions to a Virginia 529 plan of up to $4,000 per account per year are deductible in computing Virginia taxable income, with an unlimited carryforward of excess contributions."

My understanding of that is that an "account" is an account number.  So if a child has 5 different account numbers because they're invested in 5 different portfolios (Intl, Total Stock Market, Aggressive Growth, etc, etc), that means you can deduct $20,000.  If a second child also has 5 different portfolios, that means you can deduct another $20,000.  If you're married filing jointly, I don't see how you can deduct $8,000 per account number. 

[Laura33]

@Buffalo Chip - Here's what the VA 529 page says about tax deductibility:

"Contributions to a Virginia 529 plan of up to $4,000 per account per year are deductible in computing Virginia taxable income, with an unlimited carryforward of excess contributions."

My understanding of that is that an "account" is an account number.  So if a child has 5 different account numbers because they're invested in 5 different portfolios (Intl, Total Stock Market, Aggressive Growth, etc, etc), that means you can deduct $20,000.  If a second child also has 5 different portfolios, that means you can deduct another $20,000.  If you're married filing jointly, I don't see how you can deduct $8,000 per account number.

IME in MD, "per account" means per the holder, per kid.  So DH and I each have separate accounts for each of our 2 kids = 4 accounts x $2500/account = $10K tax deduction.  We could not open separate accounts with different portfolios for the same kids and have them count as separate accounts, though, because it's all stuff that we are holding for the benefit of the same kid. 

Personally, I wouldn't worry about any of this right now, though.  You are worried about covering your bills because your income dropped.  That means you want to maintain accessible cash, not lock it up in illiquid assets like your house or accounts where you'd need to pay a penalty to access it like a 529.  Now, if this reduced income were permanent, my answers may be different.  But if it's the 1-2 years that you project, you should still be following the plan that maximizes your long-term rewards, which you do by keeping as much in the market as you can (which also means cutting expenses so you draw as little as you can from that pot).

[TomTX]

Couldn't you also open a 529 for each parent as well? That's 4 people instead of 2, and I believe you can re-designate the beneficiary later on if they're close family. Verify for yourself!

[Boofinator]

@Mako52 What were your closing costs? What company did you use to refinance? I've wanted to refinance, but have been hesitant because I'm not sure this is a long-term house for me and I don't want to shoulder the high closing costs I've been able to find.