Conventional wisdom is to keep it with such a low interest rate (1.75%), but I wonder if that would be different for a retired household.
Generally speaking, this goes to risk reduction in retirement. When you are in your earning years, you are your biggest asset. If the market goes south, you can work harder, get a different job, get a raise/promotion, etc. etc. etc. to earn more to support yourself and your family. You can afford to invest aggressively, because you are your own safety net.
When you retire, however, that giant safety net largely goes away (more or less depending on how easily you could go back to work, but since the goal is not to
have to go back to work, we'll treat it as going away entirely). And that means that now your investments (and future SS, any pensions, etc.) are now your only real safety net. That means you can't afford to be quite so aggressive everywhere else -- your balance shifts more from seeking growth to managing the downside risks that could blow your plans out of the water.
Usually what that means is reducing the mandatory monthly nut. And the easiest way to do that is by having any and all debt paid off. If you think about it, having a mortgage at this stage in your life is basically using leverage to maximize growth -- you are making a bet that you will make more money in the market than you will lose via the mortgage interest. It's basically the same as borrowing money to invest. And leverage is fantastic for magnifying the effects of gains -- but it equally magnifies the effects of losses. Ergo, for most folks, going into retirement without a mortgage is the best way to help protect against downside risks.
But:
Your mortgage is at 1.75%. Holy crap. That's basically free money. You can get guaranteed-interest assets, like CDs and individual bonds, that will make you a lot more than that. We currently have 6-month CDs at something like 4.5%-5%. There is literally no risk involved in putting your money in those kinds of instruments (well, unless the banks all fail, but then we're all screwed anyway). If I were you, I'd be putting the amount it would take to pay off the mortgage into those kinds of guaranteed-return assets. That way, you can continue to take advantage of your ridiculously low mortgage rate for as long as it continues to be beneficial (my math says that if you have $680K in a CD returning 4.75%, that will earn you over $20K/year -- that's not insignificant!). Then, if and when interest rates drop significantly, you can take that money and pay off the mortgage.
My advice is also shaded by your note that you might go back to work. If you're still in your working years, then you an afford to take more risks with your investments, per the above. I'd hate to see you lose the benefit of such a ridiculously low mortgage rate because you wanted to take a year or two away, if in the end you were going back to work.
Finally, just focusing on a fixed withdrawal rate is overly conservative in your situation, because (1) you will be eligible for SS in less than 10 years, which would cover a chunk of your annual expenses, and (2) your mortgage will eventually be paid off no matter which option you take. I suspect if you looked at a more detailed calculator that took those into account, you'd find that you can easily retire now and never have to work again, no matter what you do with the mortgage. So congrats!
