Author Topic: Dont be fooled by Illusion of Past Performance  (Read 4315 times)

soccerluvof4

  • Walrus Stache
  • *******
  • Posts: 5444
  • Location: Artic Midwest
  • Retired at 50
    • My Journal

dragoncar

  • Walrus Stache
  • *******
  • Posts: 8796
  • Registered member
Re: Dont be fooled by Illusion of Past Performance
« Reply #1 on: February 25, 2014, 02:18:02 PM »
Can a statistics guru verify the "random chance" figures below?  My gut tells me they are wrong:

Quote
The latest scorecard, published in December , analyzed the subsequent performance of the 837 domestic stock mutual funds that produced top-half performance for the five years ending September, 2008. How many of these funds were able to retain their top-half status for the next five years? Random chance alone would lead us to believe that 50% (419) of the funds would do so. Yet S&P reported that only 43% (363) managed to retain top-half status.

The S&P Persistence Scorecard also tracked top 25% funds for the five years ending in September, 2008. The 419 funds that were able to generate this level of peer outperformance might be benefiting from skillful managers. We would expect 25% of these funds (105) to remain in the top 25% for the next five years by random chance alone. Yet only 22% (94) were able to do so.

In both cases, the number of repeat winners is less than what we would expect from random chance. Therefore, it is up to the proponents of active management to provide evidence that repeat success was produced by manager skill rather than luck.

edit:  For example, if we assume each fund has a 50% chance of being in the top half each year, the "random chance" that any fund will be in the top half five years in a row would be 3.125%.  Thus the fact that so many funds stayed in the top half for five years seems quite impressive.
« Last Edit: February 25, 2014, 02:20:20 PM by dragoncar »

KingCoin

  • Pencil Stache
  • ****
  • Posts: 783
  • Location: Manhattan
  • Achieved FI @ 30
Re: Dont be fooled by Illusion of Past Performance
« Reply #2 on: February 25, 2014, 02:26:53 PM »
Can a statistics guru verify the "random chance" figures below?  My gut tells me they are wrong:

Quote
The latest scorecard, published in December , analyzed the subsequent performance of the 837 domestic stock mutual funds that produced top-half performance for the five years ending September, 2008. How many of these funds were able to retain their top-half status for the next five years? Random chance alone would lead us to believe that 50% (419) of the funds would do so. Yet S&P reported that only 43% (363) managed to retain top-half status.

The S&P Persistence Scorecard also tracked top 25% funds for the five years ending in September, 2008. The 419 funds that were able to generate this level of peer outperformance might be benefiting from skillful managers. We would expect 25% of these funds (105) to remain in the top 25% for the next five years by random chance alone. Yet only 22% (94) were able to do so.

In both cases, the number of repeat winners is less than what we would expect from random chance. Therefore, it is up to the proponents of active management to provide evidence that repeat success was produced by manager skill rather than luck.

edit:  For example, if we assume each fund has a 50% chance of being in the top half each year, the "random chance" that any fund will be in the top half five years in a row would be 3.125%.  Thus the fact that so many funds stayed in the top half for five years seems quite impressive.

Not sure what you're quibbling with. They're making the assumption that random chance dictates 50% of the funds will have performance in the top 50% and 25% of the funds will be in the top 25%. Seems tautologically true. They're additionally arguing that if returns are random, a fund manager will be equally like to end up in the top 50% or bottom 50% (or 3 times more likely to end up in the bottom 75% as the top 25%).

Hamster

  • Pencil Stache
  • ****
  • Posts: 623
Re: Dont be fooled by Illusion of Past Performance
« Reply #3 on: February 25, 2014, 02:29:39 PM »
As for the statistics, being in the top half for an aggregated 5 year period is different from being in the top half every year for 5 years in a row. I think that is where the confusion comes from. They must be speaking about the first scenario.

PeteD01

  • Bristles
  • ***
  • Posts: 415
Re: Dont be fooled by Illusion of Past Performance
« Reply #4 on: February 25, 2014, 02:31:38 PM »
The expected random distributions are correct.
For a strictly random ranking one would expect that last years winners would be equally distributed around this years median.
Makes one think that whatever worked in one year is less likely to work the year after but the managers persist in doing what worked the year before. Has probably something to do with the managers having no clue what it was that worked the year before...

Peter

KingCoin

  • Pencil Stache
  • ****
  • Posts: 783
  • Location: Manhattan
  • Achieved FI @ 30
Re: Dont be fooled by Illusion of Past Performance
« Reply #5 on: February 25, 2014, 02:37:57 PM »
Makes one think that whatever worked in one year is less likely to work the year after but the managers persist in doing what worked the year before. Has probably something to do with the managers having no clue what it was that worked the year before...

Yes, the more interesting question is whether the nonperformance is statistically significant or not.

I'd complain that using median isn't really instructive. A manger who massively outperforms over a 5 year period, but then only slightly underperforms in the next 5 year period, may still be a very good manager.

dragoncar

  • Walrus Stache
  • *******
  • Posts: 8796
  • Registered member
Re: Dont be fooled by Illusion of Past Performance
« Reply #6 on: February 25, 2014, 03:08:19 PM »
As for the statistics, being in the top half for an aggregated 5 year period is different from being in the top half every year for 5 years in a row. I think that is where the confusion comes from. They must be speaking about the first scenario.


Right, the phrase "retain their top-half status for the next five years" led me to believe it was the latter.  If a fund drops out of the top-half, and then regains it, I do not consider that having "retained" it's top-half status for five years.

KingCoin

  • Pencil Stache
  • ****
  • Posts: 783
  • Location: Manhattan
  • Achieved FI @ 30
Re: Dont be fooled by Illusion of Past Performance
« Reply #7 on: February 25, 2014, 03:21:49 PM »
Right, the phrase "retain their top-half status for the next five years" led me to believe it was the latter.  If a fund drops out of the top-half, and then regains it, I do not consider that having "retained" it's top-half status for five years.

Yeah, the language is kind of ambiguous.

warfreak2

  • Handlebar Stache
  • *****
  • Posts: 1140
  • Location: UK
    • Music by me
Re: Dont be fooled by Illusion of Past Performance
« Reply #8 on: February 25, 2014, 03:36:29 PM »
I read it as, those funds had the best return after five years. I think even proponents of actively-managed funds would agree that performance over individual years isn't a good measure. If you only wanted to invest for one year, the stock market would be an inadvisable choice, whoever the fund manager is.

dragoncar

  • Walrus Stache
  • *******
  • Posts: 8796
  • Registered member
Re: Dont be fooled by Illusion of Past Performance
« Reply #9 on: February 25, 2014, 04:25:59 PM »
I read it as, those funds had the best return after five years. I think even proponents of actively-managed funds would agree that performance over individual years isn't a good measure. If you only wanted to invest for one year, the stock market would be an inadvisable choice, whoever the fund manager is.

Well the scorecard is called "persistence" not "performance".  To me, persistence refers to continuing performance, not just the end result.

But I agree with you that they are really just looking at performance at the end of five years.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28056
  • Age: -999
  • Location: Seattle, WA
Re: Dont be fooled by Illusion of Past Performance
« Reply #10 on: February 25, 2014, 10:10:07 PM »
The article was basic, but a good summary of the issues with active investing for a beginner. Thanks for sharing.
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with two kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.