You're analytical, and it sounds like he is, too. So what if you tell him that you will commit to working as long as it takes to make up the difference from the lost income (due to time off while unemployed and to a possibly lower salary)? Also, are there any cuts in the budget on your stuff that you can make, especially while unemployed? You'll get your hair cut 50% less often and for cheaper, you'll give up the monthly cleaning lady and wash the cars yourself, whatever.
It you make $100k now (number chosen for easy math) and end up being out of work for 6 months (3 without job searching, and another three to find a job), that's ~$40k less money coming in. (6 months, but after taxes, and presumably slightly lower costs due to not commuting and more time to cook, etc., but run the real numbers for yourself.) Then you find a new job making $75k. (Of course, the new job could pay the same, or more, but run a worst case.) If you'd planned to work another 5 years before you quit, that's a difference of ~$100k (25k/5 years, but after taxes). So in total, you are coming up $140k short, so you'd work an extra 2 years. If he hates numbers, skip to the bottom line. "I estimate this move will cost about $40k in lost income, and I'm willing to work extra time before retiring to make up that, and to make up any potential difference in income if the new job pays less. I know we had an agreement, so I need you to know that I am absolutely going to honor that agreement, which is why I will do this in a way that has the absolute least impact on us financially as possible, while still allowing me to keep my sanity. I will do whatever it takes to make up any short term difference."
Whatever the numbers are, show him that you are willing to rearrange things, and even commit to delaying your FIRE date, in order to make this happen--that you've come up with a plan to allow you to leave this horrendous job without having it affect the family finances.
Also, is there no other industry you can work in? I don't suppose your work offers free legal consultations? Heck, even a JAG might be able to tell you whether the contract is enforceable. At the very least, start looking at anything that is outside the scope of your non-compete. No need to wait 3 months to do that. You should have spousal preference for GS jobs, which isn't great, but it's something.