I find the days I need to go in for a shift I dread; and when I work both jobs I can't exercise that day, which is a real bummer.
Get out. At the very least, look for ways to change what you dislike about the situation (different manager?) and retain what you love.
I left my last company because I had an overbearing micro-manager for a director, stressful interactions with some of the people reporting to me, a giant workload, and basically no life outside of work except for complaining about work. I deliberately searched for a new position that would be easier and low-stress so I could learn to relax again.
I was "no promises" non-officially offered a blank check to stay after I handed in my notice. Nope nope nope. I felt so much happier and more peaceful after I left. As long as you're financially stable without it, leave. You can always find another option. You may also find that it's a wash financially. With the additional free time your spending may go down, or you may be able to ditch the car entirely and find alternatives, etc.
Have you considered posting a full case study to see if focusing on your expenses could yield an equivalent return to the part-time job?