What stock market are you investing in? US? US historical average returns are around 7% in today's dollars and 10% in inflation-adjusted dollars.
According to this calculator,
https://www.investor.gov/additional-resources/free-financial-planning-tools/compound-interest-calculator, you can expect $1,000,000 earning 5% per year, compounded annually, to be worth about $1,600,000 in 10 years.
At the historical US average return in today's dollars, you're looking at ~$1,967,000 in the same timeframe.
Compounding growth may be more powerful than you realize.
Mortgage terms? What rate, payments and length of payments?
If you have really good terms you should delay paying it off too early and invest the amount instead.
My own mortgage is a 2.75% fixed rate mortgage for 15 years, with 14 years to go. If I can average 10% (7% growth plus 3% inflation) over those 14 years and only pay 2.75%, it would be silly to pay off the mortgage early. There's about a 2% chance, assuming the future is no worse than the past, of it being better to pay off the mortgage early. Given that you're expecting to work another 12-15 years before your FIRE date, you'll know ahead of time if that's going to be a concern.