So what do you plan to do after you FIRE? And by that, I mean, "do" as compared to "not do"?
When you focus on FIRE as an escape from all of your daily inconveniences, you will inevitably be disappointed: you white-knuckle it through 15 years with your head down, you FIRE, and then two weeks later you still have to get up at 5:30 (it's just to go on a trip), you still have to sit through interminable meetings (it's just for a volunteer position you've taken), etc. -- and then you have a huge letdown and get angry and wonder why you missed 15 years of your life when all your problems are still there. The reality is that life comes with annoyances, and many of the things worth doing involve uncomfortable tradeoffs. Or, as that great philosopher Buckaroo Banzai noted, "no matter where you go, there you are." (Emphasis on the "you," if that wasn't sufficiently clear from context)
The best way to avoid that kind of situation is to have a vision of the stuff you want to do -- not just "sleep in," but devote your time to, give you a purpose. What important things can you offer your family and your community that you can't do now because of your schedule? Now: what are the negatives that go along with that? E.g., volunteering can be a part-time (or full-time) job in and of itself, with all of the annoyances that go along with that; you'd hope that it feels more worth it when you are contributing to something you really care about and believe in, but that doesn't mean that the annoyances aren't there. Work to build a more realistic mental vision of what your post-FIRE life will actually be.
(I have to admit, this was one of the most depressing realizations I've ever made -- you mean my life is always going to involve a schedule and time constraints and all the crap that I hate now? Yes, because I am not happy puttering 24/7, and so to be actually content with my life, I need to sign up for things that will make me get out of bed and schedule my planned long trips carefully and all that. Dammit)
The other thing I will recommend is to stop looking at FIRE as sacrificing everything fun now for this wonderful idealized future life. The next 15 years of your life matter. It's the only time you will be that age; the only time your kids will be that age; the only time you and your SO will spend together in this place at this age with this particular family. The point of FIRE is to maximize overall life happiness. So if achieving that goal means missing all the fun for the next decade and a half, then boy, that thing you're working and saving for better be really fucking awesome. And really, nothing can live up to that expectation.
So why not take a bigger-picture look at your life? What are you missing out on right now? How can you get some of that back into your life? Can you find a more appealing job? Can you go part-time to enjoy the ride a little more? Is there a hobby you can do, either individually or with SO/kids to really appreciate your time together? A lot of people do FIRE a lot of different ways; some are willing to work insane hours in insane jobs for a short time to be able to never work again; some cut back on work in the short-term even though it moves the FIRE date back; some people do a bunch of mini-retirements or take long trips/breaks, even though it means working longer; some people work at a career until they get to a certain financial point and then plan to take on part-time work or do passion projects to continue to bring in a little income; etc. So be creative. Figure out which version of that will work best for you and your family to give you the best overall ride.