The majority of reasonable asset allocation policies except some very simplistic (not necessarily bad) ones include some amount of small and mid cap stocks. So either a total stock index fund is needed or some combination of large cap (like the S&P 500) and separate small (and maybe mid) cap funds. There is no fundamental difference between the two solutions.
Those with a long time horizon should assess whether increasing the proportion of small and mid (vs. large) cap stocks relative to the composition of a total stock market index would make sense for them. If the answer is yes (as it is in my opinion for most people), you'll need multiple stock funds even if you go with a total stock market fund.
Rather than pitting total stock market index and S&P 500 funds against each other, a more useful approach is to decide on some asset allocation policy that works for you, and then see whether a total stock market fund or an S&P 500 fund makes it easier to stick to your goals given other constraints. Those constraints could be the selection of funds available in a retirement account, how some other funds you might like to have skew the balance between large, mid and small cap stocks, or how you can avoid wash sales if you do tax loss harvesting, and so on.
Unless you wish to follow some very simple asset allocation policy, it makes sense to research the composition of various funds because it's rarely the case that if a fund calls itself such and such (e.g. mid cap), then it invests in almost nothing else. Morningstar.com, for example, offers some useful tools that help analyze portfolios.