I did it on a much larger scale. I was making $25/hr for a position that required only experience, no certifications or degrees. I busted my butt to get to that position, and ended up absolutely hating it. Now, two years later, I'm still making way less, but I'm so much happier.
Today, I had to skip out in the middle of work to go get my child from school. She ended up getting the stomach flu. Also, I now have more time to focus on learning how to code – something I've wanted to do for so long, and something that will lead to an income of ~2x what I was making two years ago.
All that said, if my wife and I were in your position, I would ask, How close are we to Financial Independence? If the answer is pretty close (within a couple years), I'd stick it out with the job. But even then, I wouldn't knock myself if I chose to go for something that might mean higher satisfaction and lower stress, especially if I'm still making over $100,000/year.