I guess my question to you would be how far away FIRE is for you? If it's just a few more years, then the pay cut may not be worth it. You can do whatever you want after you FIRE, regardless of pay. For example, I don't adore my job, but I don't hate it and recognize this is the fastest vehicle to FIRE.
But if it's 10+ years and you're going to be miserable, than you might want to seriously consider changing careers. Being miserable can affect your performance, and ultimately your pay, as well.