You're not alone. We will likely FIRE in March 2018 but I often have sleepless nights thinking about sequence of returns risk and the fate of the ACA. I often wonder if I'll have the courage to actually quit a high paying job with good benefits. There are some days when I feel optimistic about early retirement but then the stress/anxiety really sets in when I read various early retirement blogs and people in forums who don't count on social security in their retirement plans (they call it "gravy"). They don't rely on retirement planning tools like cfiresim and firecalc and prefer to use various self created spreadsheets. They feel that 3% is the new SWR but even that's probably not sustainable. They might even have a spouse that will continue to work and will provide health insurance.
So yes, I feel a lot of stress and anxiety because we could very well be retiring toward the end of the 2nd longest bull market in U.S. history, we're relying on ACA (with subsidies) for our health insurance, we use planning calculators like cfiresim, firecalc and Fidelity's retirement analysis, we are counting on at least some social security as part of our long term income and our withdrawal rate will be 3-3.5% annually.
But with that said, if you worry too much about market risk and legislative risk, taxes going up, social security going away, health insurance reverting back to pre ACA days and if you compare yourself to other FIRE bloggers who probably create elaborate spreadsheets to account for varying levels of inflation for different expenses, you will never feel like you can retire. You will feel like your stash has to be twice as big as it is now. This is how you fall into OMY syndrome.
If you over save and over plan, you lose your time and part of your life in the process and you'll never get that time back. The time spent doing work that you don’t enjoy so you can have a stash big enough to overcome every possible scenario has a cost.
If you feel like you have a good plan in place and you're not just winging it, if your expenses are dialed in with some room to increase spending if necessary, if you'll remain flexible during a long retirement and possibly bring in some part time income someday (maybe in a job you really enjoy), then I say leave that job you don't like, don't fall into OMY syndrome.
(sorry that was so lengthy but I just ran with it. It was just as much for me as it was for you so thanks for the question)