So, first, are you actually sure you cannot live on your husband's salary alone once the baby comes? Single-income, head of household, $72K, with a dependent is going to give you a pretty significant cut on your income taxes compared to what you're paying now as DINKs. Pop over to the IRS website and run some tax calculations to start -- I spent 30 seconds on an internet calculator, and it said total tax owed of about $5K, dropping down to under $3K once you add a kid (I obviously don't know your state/local tax situation; obv., NYC would be more than Houston). So if you're sticking to that $65K annual spend, you guys may already be good, depending on how much he puts into his 401(k).
Second, what you actually need to do is track your expenses, religiously, over the next year or so while you're planning for this major life change. There are many, many categories that you have grouped together/omitted, and those can add up. Track what you are actually spending so you know where you are and what you need.
Third: expect things to change pretty dramatically when that little one comes along. Right now, you enjoy spending $1K on nice restaurants and fancy ingredients to cook at home; two years from now, that might be $75 at Chuck-E-Cheese. ;-) The nice thing is that if you do quit your full-time job, you will have time at home that can allow you to optimize your spending even more (obviously not with a newborn, when you should be taking every possible moment to sleep, but by the time the baby is 6 months or so, you'll probably have some energy back and be wanting to dive into something useful). With your husband's income and your big pile o' cash, you have plenty of room to relax a bit and figure things out.
For the $600K, I'd estimate how much you want available as an emergency fund, for the baby expenses, and then to cover any delta for the next say two years after the baby is born. Put that in savings or CDs -- you can't afford to lose the value of that. Then invest the rest, following whatever your normal investment plan is.
Also: nonononono, you cannot take $4K out per month! Never make plans based on an assumption that the market is going to continue to be a bull market for the foreseeable future! You're trying right now to protect your downside risk: can you quit your day job and avoid having to go back unless/until you want to? So you need to be focusing on the worst-case scenario -- the things that could derail that plan.
Normal rule of thumb is that you can withdraw 4%/yr without running out of money. Obviously, the lower you keep that %, the higher the chance that your money can last forever. So, no, you currently don't have enough to fully retire forever -- but luckily, your DH doesn't want to! And his salary will cover, at a minimum, the vast majority of your routine expenses!! That will then allow the rest of that giant chunk of money to continue to grow. So depending on your real expenses and your DH's employment, you may never need to work another minute or invest another dollar!
Look at it this way: you could put that $600K in cash and cover all your family expenses for 8-10 years, even if your husband is fired tomorrow. That buys you a metric shit-ton of flexibility to manage your future however it works best for you. So congratulations for putting in the work to build a business and earn yourself such a great payday that now allows you to do whatever the hell you want.