The interest you ‘save’ is only one side of the coin, and a fairly warped one at that (more about that in a minute). The other is the opportunity cost - what that money could have done for you if invested in something different.
Given that you refinanced to a 15y last year, I’m going to venture that your mortgage rate is 2.x%. If that money was just sitting around as cash under your mattress (or in a 0% checking account) it would be very straightforward - putting it towards a mortgage saves you in interest payments. But there are other options on where to put your money, and those are likely to earn more than 0% (some guaranteed, some just highly likely).
As for your math, what you are missing is inflation. Whenever mortgage pay-off calculators show how much interest you will pay over a given time period, they almost always ignore inflation: a dollar in inflation paid today is equal to a dollar paid several years down the road. We know that’s almost certainly not going to be the case. In your example, the interest saved four years from now will likely be discounted 10-14%, depending on whether inflation goes over the next few years. All of which means the ‘guaranteed returns’ will be much lower in real (inflation adjusted) terms than the yield on your 15y mortgage. All because you are borrowing at a fixed rate set in stone a year ago.
Here, an inflationary environment becomes your friend. If we are going to see “above average” inflatino (which most people read as >2.5%) as many suggest, the ‘return’ you get on paying down your mortgage will be substantially less, and quite possibly negative.
Finally - before I get flamed by the POYM crowd, I don’t think it’s a bad move on your part. As you’ve said you’ve maxed out your tax-advantaged accounts and have a sufficient E-fund. Reducing your expenses by axing the mortgage has its own benefits. But calculating what ‘the math’ is requires some knowledge of future conditions and a comparison of what your opportunity cost is.
