We've been grappling with a similar dilemma, though our kids are older (8,9, and 12). If we kept our noses to the grindstone, we'd be FIRE in 5-7 years. But in 5-7 years, my kids will be heading off to college or teenagers who would rather hang with their friends than their parents, so we have made the conscious choice to slow things down, under the assumption that time right now is more precious than time in the future will be where the kids are concerned.
I wouldn't be making this decision if I didn't already have good savings, and confidence that we can live on less income. It sounds like you have decent savings, and are doing what you can to reduce expenses. I recently went to a reduced workweek (75%), resulting in about $1200 less in take-home pay each month. I have cut monthly expenses by $2600 and, if we can live this way for six months, then I will feel comfortable cutting hours further.
Good luck with this decision!