Would I FIRE in your shoes? Not likely with the assumptions I'm making. As your spending has apparently been going up from what I'm reading here, I'm going to assume that the $80-100k should just read $100k/year. That's spending, I'm assuming taxes not included yet. We'll get back to that though.
I popped your numbers into FIRECalc -$78.4k/year spending (removed the mortgage from that amount assuming you're paying taxes/insurance separately), 50 year retirement (probably should use longer as that only takes your wife to 85, but the numbers generally don't move much after that), $21.6k/year spending for the next 19 years for the mortgage PI, $24k pension starting in 2038. For that scenario, FIRECalc gives an 82.3% success rate. It doesn't take into consideration any further lifestyle creep, taxes, or other potential increases in spending moving forward which would make those odds worse
IF, on the other hand, I'm wrong and you're spending is actually $80k/year including taxes and the mortgage, AND you can both commit to not increasing it at all in the future, then FIRECalc says you're good to go even adding in a 10% effective tax rate. Oh, and this plan probably goes to hell if your relationship doesn't last forever (even retired people get divorced, sometimes all that free time can hurt a relationship studies have shown).
Alternatively, you might consider just leaving the job you seem to hate and trying to find something you would enjoy doing (even if for less money) until the numbers are a bit more conservative.