If there is in fact enough stache to retire today into a reasonable lifestyle that spouse 1 would be 100% comfortable with, but with which spouse 2 would feel overly constrained, that does change things.
But since college savings were mentioned, unless spouse 2 if fine with not doing that (or with whatever has been put aside up until now), it seems like that's not quite the case.
And even if it is the case, to me, marriage is about compromise. I don't just get to quit when I'm comfortable, regardless of what DH about that. I'd likely take my number and his and average them, and work until we were at a stache that would allow that amount If he wasn't willing to meet in the middle, then my proposal would be that I'd work until the middle and the rest would be on him. So if I wanted 40k and he wanted 80k, I'd work until we got to a number that would allow us 60k in spending money and then quit. Of course this would be a conversation and I'd be open to other options and to his input, but that's where I start.