I'm a lot less worried about fluctuations in the stock market than I am about changes in tax treatments.
The 2% FICAl tax cut that was instituted last year is set to expire, and nobody is talking about saving it. Instant 2% loss of income compared to last year, and even though I support the idea it still stings a little.
I live in a state with no state income tax, and such states have historically had a federal tax deduction for state sales tax to offset the benefits that people from income-taxing states get from deducting their state income taxes. Unfortunately, it looks like the state sales tax deduction (but not the income tax deduction, just to rub it in) is expiring, which is going to cost us several thousand dollars per year.
I'm not so worried about the Bush tax cuts, as I suspect they will stay for people in my income bracket. Likewise I don't care much about the expiration of unemployment benefits or the doc fix, as they don't affect me personally. But there are a handful of other taxes (for healthcare and capital gains, for example) that are probably going up, and that complicates my planning spreadsheets.
The stock market taking a 5 or 10% dive? Just whistles in the wind, easy to overlook if your time horizon is long enough.