If you have a 457b, I assume you are a State for City government employee. In that case, I think the big question is how confident you are in your employer's solvency.
If you are very confident in your employer's solvency, then you are in a great place. Given that you should be able to live comfortably off of your pension alone, I think your plan to invest with the idea of covering future losses to inflation and exceptional expenses is wise. And you can afford to invest conservatively if that makes your spouse happy, because you don't have to maximize your income to meet your expenses.
If you are not entirely confident in your employer's future solvency, I think the situation is a lot trickier. If your pension were to disappear or be cut substantially, that $650K would suddenly shift from being a nice cushion and a fund for emergencies and luxuries to one of the pillars of your retirement. You don't mention Social Security, but I think it is still likely to be a useful boost to income, and something to factor (positively) into your equation.
I retired from the Feds a few years ago (at about the same age you'll be retiring), and I don't think you've missed anything in your plan. At the risk of derailing your thread by provoking the "don't pay off your mortage folks," I have to say that there's a lot to be said for entering retirement with a paid off mortgage. In theory, it may be mathematically suboptimal, but your expenses are the part of your retirement that you can control. Income is a lot tougher. Personally, I'd try to live off my pension alone for as long as possible and let my investments grow for as long as possible with as aggressive a portfolio as you and your spouse feel comfortable with, given your relatively young age and long retirement horizon. By the time inflation is taking a serious bite out of your pension, you should qualify for SS.