I think you are confusing the variance of returns over a statistically insignificant period of time (such as a year) with the variance of the overall returns over a period such as 30 years. If you follow a true value oriented market approach where you buy businesses (which are what stocks represent) when they are on sale and which have solid fundamentals (i.e., are a good business) then you are virtually guaranteed to far out-perform the broad market which consists of all businesses. This is just common sense not rocket science.
There are no certainties in the market or in life. Your vast cooking skills that you've spent many precious hours to obtain may end up yielding naught if you get hit by a bus tomorrow or choke on your chicken soup and die. Would we base the value of those cooking skills on an outlier statistic? That doesn't seem wise.
I will preface this by saying that I don't think it's 'wrong/bad' to invest in individual stocks, a portion of my portfolio is also in individual stocks and I accept the associated risk in order for the opportunity to outperform. But I have a
significant portion of my accounts in index funds/ETF's. I am aware of the many ways one can value a company as I do it as part of my profession, as part of managing my own investments, and learned how as a part of my formal education. I'm not claiming I'm an investing/valuation genius but in general I know the ropes and you are very right it's a valuable tool in the investing arsenal.
This is the part of the statement I disagree/take issue with:
'If you follow a true value oriented market approach where you buy businesses (which are what stocks represent) when they are on sale and which have solid fundamentals (i.e., are a good business) then you are virtually guaranteed to far out-perform the broad market which consists of all businesses. This is just common sense not rocket science.'
1- In a perfect world where you have all available information available to you (as an independent investor) this is somewhat 'true'. The part you are missing: you DON'T have access to all the information, no matter the amount of research you do. This is why when a company offers a merger/acquisition tender offer for a company, you are allowed a due diligence period where you are able to shine a flashlight into every nook and cranny, and not just a 'look at the publicly available data'. A vast amount of relevant and important info is NOT available to a retail investor, and as such you have no idea what skeletons are hiding in which closet. Broad market example: Enron. Viewed by many is a good solid business, the stock had performed admirably, people on Wall Street were high on it. Everybody was making money and patting themselves on the back about what great solid valuation business picking skills they had. Until the depth of corruption became apparent and the whole house of cards collapsed. No retail investor could uncover that scandal beforehand with any amount of 'diligent research'. Tons of Enron's own employees, some even fairly high up, had no idea. That can happen at
ANY company in America...'Ah well I will just avoid companies with unknown scandals'...to that I say...OK.
2 - nobody has a crystal ball. Sometimes you can see the downfall of a company or industry coming (many astute people (not saying I am one, but they exist) saw the impending financial collapse in 07/08, some profited quite heavily)...but sometimes freak events happen and you can't see it. The most recent example to come to mind is the BP Macando spill. A well blows up one day and there are billions in damages....who could see that coming. In all other regards BP was functioning as well as any other oil company and had been doing fine. If you can tell me that a retail investor can discover lax safety protocols in a company that may have an incrementally higher chance of leading to such an event, I'll say it is technically 'possible', but lets be realistic. It's probably not, or at the very least, nobody puts in the effort to find out. That's one example. Freak events can happen in any industry to any company. Ask the drug companies who made the next life saving pill only to find it causes cancer 15 years later and drowned in litigation.
So I reject your assertion that by doing enough homework and knowing how companies are valued, you are 'virtually guaranteed' (your words) to outperform a broad basket of American companies. You MAY outperform...and you MAY not. Yes SOME people will outperform based on their skill and also because they were lucky enough not to have anything unforseeable blow up in their face. SOME people will have great skill and get the bad end of the draw and have the tar kicked out of them.
Thus why index funds are not purely the 'lazy/average' investor's avenue of choice as I feel has been implied by some in this thread, but rather an avenue for the investor who seeks the best combination of expected risk/return. When an Enron or Bear Stearns etc goes down, it's insulated by the diversification.
I hope you don't find this to be a personal attack. It is not meant to be, just for clarification. I just have issues with advocating 'virtually guaranteed'...nothing is virtually guaranteed, but picking individual stocks certainly is not.