Well, what else could you do with the money? Assuming you have a buffer, are you leaving an estate? Can you give more to charity? Take another trip?
It's too early to say that we are spending less than planned long term, if that was the case I could find some ways to increase mindful spending and more charity. Right now we've had one year of much lower than planned withdrawals, but I would say there's a possibility of some years of higher than planned withdrawals in the future. Wondering how to use the lower withdrawal years to best plan for and mitigate the possible but unknown future higher withdrawal years.
If it was a bad idea to keep piles of cash sitting around before being retired, it's still a bad idea. Money that you don't use should do what money that you don't use has always done . . . stay in the investment accounts. If your asset allocation is way off, re-balance it.
The difference is once you start retirement, you have to worry about sequence of returns risk, and what I'm wondering about I suppose could be called sequence of withdrawal. I'm wondering if you can mitigate any risks somewhat if it seems your withdrawal needs are very uneven.
I'll try making up an example of what I'm worried about. Person A and Person B both retire with one million dollars, and plan to spend $40k annually.
Person A ends up withdrawing $40k each year as planned.
Person B withdraws $10k the first year and $70k the second year.
Say markets go down 10% the first year and up 20% the second year (both people withdraw their money on the first day of year for simplicity)
Person A ends up with $3,600 more invested at the end of the two years, even though they took out the same total amount of money and had the same market returns. In this case, Person B could have forced their withdrawals to be the same as A (though it's only with hindsight we know that would have been beneficial).
So going through that exercise, I see I had to make a pretty extreme example of uneven withdrawals and market swings to make a small difference over two years. Not sure if it's a sign of nothing to worry about at all, or something that could possibly build up over time. Also I know you can just as easily make an example where Person B comes out ahead because of the uneven withdrawals.
Leave it invested. The extra growth will add extra padding so that if the future does end up being worse than the past you'll be that much more likely to still be okay. Then if things turn out to be just like they have been you'll be able to make some large donations, or raise your spending a bit, later in life!
This is probably right on average. I'm not sure how to make the risk adjusted choice between trying to even out spending if possible and needed as opposed to just leaving as much invested as possible and hope you don't run into a bad sequence of returns and withdrawals. There probably is no way to answer that prospectively, but was wondering if anyone had examined it in more detail.