Your expenses are currently $54,000/year, which seems very high but I'll run with it. If you intend to keep up that level of spending, you would need 1.35mil in total assets, assuming a 4% initial withdrawal rate. This rate may be high or low for your comfort level, but I'll stick with it to run the following numbers.
Your total assets are 969k, excluding the college account. This means you can't retire yet regardless of how your money is allocated into tax shelters. You could retire today if you could cut your monthly expenses from $4500 to $3200. I'd take a hard look at your budget and decide if every item in there is worth continuing to work for that many more years.
Alternately, you might consider downgrading from your paid off current 950k house to a half million dollar house, and using the surplus to reach your 1.35 million target amount and retiring immediately. If not, keep in mind that you are only continuing to work in order to keep that current house instead of one that costs $500k. You could keep your current expense level if you made that one housing move.
If you wanted to do this immediately, your 401k money is accessible through rule 72(t) withdrawals. Go to
http://www.dinkytown.net/java/Retire72T.html and plug in your own numbers, but it looks to me like a 47 year old with 675k in her 401k can withdraw 23.4k/year using 72t, penalty free. Your 50 year old husband with 150k can withdraw another 5.5k/year, giving you a total of about $29k/year from your 401k right now, without paying the 10% early withdrawal penalty.
In order to hit the $3200/month expense limit that your current assets might reasonably support, you need to come up with another $800/per month. This can you easily do with the 95k in your brokerage account, depleting it by 800*12*5=48k over five years.
Why 5 years? Because that is the amount of time you would need to build the famed 5 year pipeline of rollovers from your 401k to your IRA, at which point that money can then be withdrawn penalty free as well. This is a second avenue of tapping your 401k before age 59.5.
If you instead were to downsize to a 500k house and pocket the difference to hit your 1.35mil target needed to maintain your current 5400/mo expenses, you will take a bit of a hit because the capital gains on that purchase will eat some of the profit. You would probably have to move to an even cheaper house, say a $400,000 rickety shack in a slum somewhere, in order to have enough money to pay the taxes.
Alternately, you can just continue to work and build towards that 1.35mil number and then not worry about it. I think that with 90k in a brokerage account and rule 72(t) withdrawals, you'll be fine. I'd consider continuing to pump the 401k, on the thought that 72t will pay a significant portion of your expenses (and an ever larger portion as you get older) and doing a five year IRA rollover will allow you access to the rest of your 401k before normal retirement age.
Wow, that was long and rambling. Sorry.
