Personally, I buy the whole market (VTI). I like having some exposure to small caps and am too lazy right now to figure out the ideal asset mix (for me). One of these days I might get around to figuring out if I need more or less allocated to the various domestic equity sub-classes. At that point, I'd shuffle things around.
If you get down into the weeds, there are a few tax implications around the options. Since companies enter and leave the Dow and S&P, there can be realized gains. If you own the whole market, then index membership doesn't really matter. Of course, ETFs can help, as can holding in tax-advantaged accounts.
An article to read and think about:
http://www.zacks.com/stock/news/170627/3-tax-efficient-etfs-for-your-portfolio(Found via quickie search, doubtfully the best article on the subject, but a starting point.)
It discusses turnover and tax cost ratio and points out a few ETFs in light of those (I wouldn't recommend those ETFs to the "average" investor, since they should not be working that hard on investing).