I'm having a hard time understanding the cash flows involved here.
I see gross salary of about $975,000 per year. Pre-tax savings opportunities at this income level (for regular wage earners) are trivial percentages -- let's assume $50,000 across 401ks, HSAs, whatever else. Income taxes in California are punitive, so I assume this all translates to $550,000 in take-home pay.
We are told that living expenses ex-housing are $50,000 per year, and that if the wife works another year at her $500,000 per year job the family will be able to save an incremental $100,000 -- I assume after tax. When the wife stops working, the family will be cash flow negative on the husband's $475,000 per year job. This implies that the family is spending $400,000 per year, after tax, on housing. ($550,000 after-tax take home, less $50,000 in living expenses, less $100,000 taxable savings). Maybe it's more like $350,000 or $300,000 per year, if there's another $50,000 to $100,000 in after-tax savings and/or stealth luxury living expenses that have somehow been unaccounted for (contributions to kids' college funds? frivolous spending that the wife conveniently overlooks?). All this implies a monthly mortgage payment (PITI) in the $25,000 to $35,000 range.
What have you guys done? Bought a $5,000,000 house with $1,000,000 down? Are you enduring yearly property tax bills in the $50,000 to $100,000 range? Do you have any idea how insane that all sounds? No wonder you were cagey about putting down your monthly / annual budget figures by category, including housing costs. You'd be laughed out of this forum.
I know California has a fetish for plutocrats pretending to be normal middle-class Americans with mortgages 'n' car payments 'n' stuff -- dressed in hoodies instead of the Monopoly Man outfit -- but for crying out loud develop some self-awareness!
You are playing with sums of money that, in any normal / sane part of America would enable you to retire to a big house in the fanciest neighborhood in the best school district with work being 100% optional. You seem to be tying yourself down entirely on the basis that you don't want to move while your youngest kid is still in high school. I know parents sacrifice for their children, but this is taking things to a fetishistic extreme. You and your husband have been and will be sacrificing years in the office and years away from family back east just so your youngest kid can learn how to smoke pot from his existing friends, as opposed to having to do so from some new friends he'll make in a new school. School children move cities all the time. It's not that big a deal.
Yes, you plan will work. There are even a number of ways to tunnel money out of a 401k before retirement age, without paying the 10% penalty, if you need that to make the cash flows work. Look up the Roth IRA ladder and the 72t SEPP withdrawal program. I'd be a bit concerned that you are concentrating a lot of net worth and therefore a lot of risk in a single asset, but you could lop a million dollars off your existing net worth and still have plenty to retire on, so I'm not sweating it too much.
But before doing all that, on your next family vacation take a road trip across America to visit family back east, and try to reconnect with this vast country of ours and with a bit of common sense. You're post gives off the vibe of living in a very peculiar California bubble, which seems to be blinding you to what your real options and choices are.