So, basically, right now you can give up any kind of work forever and continue to live exactly the way you want to live. You can buy a nicer house and live a fancier life, which will increase your costs significantly (it's not just the purchase price, btw -- taxes, maintenance, repairs, electricity/gas, lawn care, etc. etc. etc.) and force you to continue working. So, fundamentally, the question is whether you want to trade permanent financial freedom for nicer things that tie you to another decade of work. You're the only one who can answer that question -- just be aware that working because you want to often inspires very, very different feelings than working because you have to.
I would also suggest looking at what other priorities you might have. Most specifically, are kids in your future? They can be costly little buggers -- they don't have to be the $250K over 18 years that the media portrays, but when your 4% calculation requires sticking to your current budget, any increase can be problematic. So if that is the case, then you might already have to work a little longer to build your 'stache to cover those future expenses, which means a new house will push out your working life even longer.
In the end, I'd be a little concerned that cutting your earnings and increasing your expenses at the same time is pretty damn risky. So why not do one or the other? Quit now, see if there is part-time work that you guys both want that is bringing in what you want to, see how happy you are in your current place and with the amount/type of work you're getting. And if the plan is working and you still want a bigger place in a year or two, well, your 'stache will be in even better shape to help you get that -- and your decision will be grounded in the knowledge that you're still happy to work that much and will be able to cover the increased costs without any problem. Or decide the house is a higher priority, buy it now, and then plan to stay at your current jobs for another couple of years while you figure out how much your expenses actually increase -- and use those full-time salaries to keep socking away massive amounts of $$$ until you reach the new level that you need.