It is of course possible to beat the market in a statistically significant manner. It's just incredibly rare.
You would be amazed how many people say it's impossible.
More to the point active stock picking winners beat the broad market by a little bit, but active stock picking losers lose to the market by a lot.
So if you think you're going to beat the market by stockpicking what you're essentially saying is "I would prefer a 20% chance of beating the market by a little bit in exchange for an 80% chance of losing to it by a lot."
I don't think this is generally true. Most studies I've seen show a large majority individual investor performance pretty close to the average: http://faculty.haas.berkeley.edu/odean/papers%20current%20versions/individual_investor_performance_final.pdf
Able to open the link on my computer finally. And....
You read the study wrong.
Figure one which you refer to shows returns based on quintiles of trading activity. It finds that the more you trade the less your net returns. Not surprising.
Table 4 is more to the point and it finds that the 25th percentile active trader loses to the market by 0.73% monthly (8.76 annually!)
Contrast this to the 75th percentile trader who beats the market by only 0.5 % monthly (6% annual).
The median trader also loses by 1.68 annually and the 99th percentile trader beats the market by 53% annually (impressive!) while the 1 percentile investor loses by a cringeworthy 58%!
And this is only a 5 year period studied. All data suggest that the longer the time period the less likely it is that the active investor beats the market.
All in all your study perfectly proved my original statement.
Thanks!
Well, now I've had time to finish reading the paper; I went to my son's baseball game. :-) I'll keep this civil! :-D
I don't believe I've read (interpreted) the study wrong. (NB - I referenced Figure 1, not beltim.) Actually, Figure 1 and Table IV are just two very similar ways of looking at the same data. Figure 1 is really just explaining where the difference in performance enumerated in Table IV is coming from - trading costs. Maybe one thing that confused you is that the lower quintiles (less trading) in Figure 1 perform better whereas in Table IV the higher percentiles perform better. So, you could say that the authors reversed the x axis in Figure 1. In my opinion, it would have been better to have higher performance (lower trading) in the higher quintiles rather than lower performance (higher trading.)
One note about trading costs - looking at the mean values in Table 1 you can see that the mean commission was 1.58% for purchases and 1.45% for sales. Using the mean purchase/sale values this works out to an average
$376 for a round trip! So, one thing to remember is that trading costs have decreased tremendously since the mid 90's.
And a note about bid/ask spread - looking at Equation (1) on page 780 you can see that the authors are
estimating a bid/ask spread% by using CRSP end of day bid/ask prices; they have to do this because they have no record of what the actual bid/ask spread was at the time of the trade. I'm still evaluating this formula and will have to go through Appendix A in more detail to see exactly what they are doing. However, their estimate seems suspect to me right now. Empirically, it is important to remember that the minimum bid/ask increment at the time for almost all stocks was an eighth, so $0.125. Those days are long gone.
So, comparing trading costs and bid/ask spreads between when the study was done and now shows that there is on the order of a 10X decrease in both.Now, on to Table IV. In section
C. Cross-Sectional Variation in Performance starting on page 790 the authors discuss the results in Table IV:
Though 49.3 percent of households outperform a value-weighted market index before transaction costs, only 43.4 percent outperform the index after costs. Nonetheless, many households perform very well: 25 percent of all households beat the market, after accounting for transaction costs, by more than 0.50 percent per month ~more than six percent annually!.
So, from Figure 1, I estimated that 40% either beat or match the index.
The more precise answer is that 43.4% of the households outperform the index after costs. A better, but very similar, answer. Also, note that 25% of the households beat the market by at least 6%/year! The Figure 1 data is averaged within quintiles so it is "blurred" somewhat; hence leading to the difference in the estimate. The 43.4% value jives with the data in Table IV; that means the zero crossing is at the 56.6th percentile. Since the 50th percentile is at -0.14% the data seems to be consistent.
I just noticed you made a post while I'm typing this so I'll address one comment you made there:
The lucky few who beat the market beat it by a lesser degree than those who lost to the market lost to it by.
Well, it's not possible to make that statement based off of the data in Table IV. We would need to have the entire plot and then integrate the area under the curve for the lucky 43.4% of the households and then compare it to the (negative) area under (over?) the curve for the unlucky 56.6%. The best investor beat the index by 580.2% annually while the very worst investor underperformed the index by -20.85%/month (not sure how you do that for a whole year!!). So, some very interesting things are going on in the top and bottom 1% of the binomial distribution.
And, to cover both sides of the story, here is what I consider the biggest "hole" in the paper. While the authors try to say that this study is good in up/down markets because 20 of the 72 months were down months the 1991-1996 time period was mostly very positive. Here is
S&P500 for 1991-1996. All six years are positive and 1995-1996 were the first two years of the Internet bubble. So, everyone was a genius!! :-D Maybe they should redo the exact same study for the next six years now that helicopter Ben has landed.