Isn't this more of an investment order question? The way I look at it, I have a magic FIRE number that covers our monthly burn sans mortgage payment. Let's say, for purposes of argument, it is $1m invested. Let's further assume that I am locked in at a modest interest rate, in my case 3 3/8 %, for about another decade, and further assume my LTV sits at about 50%. In this situation, while I am accumulating, it makes absolutely no sense to prepay the mortgage instead of investing toward my FIRE number. I just pay the mortgage payments as they come and shovel as much cash as possible into index funds.
Now, once I hit the FIRE number, the calculus is different. The goals I set for my money will also be different. I want to live off the stache. By my estimates, I will have 5 years left on the mortgage at that point, LTV will be in the vicinity of 25%. More than likely I will work just long enough to stuff some money into a brokerage account until that balance exceeds the payoff, then I will kill the mortgage and FIRE. Now, I could continue to dick around and arbitrage the remaining balance against what I am making on the brokerage account until the mortgage pays out on its own, but why complicate matters? Sometimes, the juice is not worth the squeeze.