Author Topic: Backtesting Robo Advisor Portfolios  (Read 5573 times)

coplar

  • 5 O'Clock Shadow
  • *
  • Posts: 11
Backtesting Robo Advisor Portfolios
« on: July 14, 2017, 08:03:28 AM »
I recently decided to start using a robo advisor for some of my taxable investments. I decided to do a quantitative comparison between them in order to choose one.

Summary:

1. I backtested asset allocations of Betterment, Wealthfront, and Wisebanyan portfolios at asset allocations of 90% and 50% stocks, from 2003-2017.
2. They all perform roughly the same as a Vanguard 3-fund portfolio matching their asset allocation.
3. The difference in performance may not be worth the 0.25% management fees charged by Betterment.
4. Despite this, tax loss harvesting and ease of use may warrant using one of these advisors.

The Numbers:

2003-2017 Average Annual Returns:

Betterment 90% Stocks: 9.40%
Wealthfront 90% Stocks: 9.52%
Wisebanyan 90% Stocks: 9.31%
Vanguard 3-fund 90% Stocks: 9.11%

Betterment 50% Stocks: 7.50%
Wealthfront 50% Stocks: 7.38%
Wisebanyan 50% Stocks: 7.66%
Vanguard 3-fund 50% Stocks: 7.25%

St. Dev:

Betterment 90% Stocks: 14.42%
Wealthfront 90% Stocks: 13.90%
Wisebanyan 90% Stocks: 13.67%
Vanguard 3-fund 90% Stocks: 13.07%

Betterment 50% Stocks: 8.41%
Wealthfront 50% Stocks: 7.62%
Wisebanyan 50% Stocks: 8.43%
Vanguard 3-fund 50% Stocks: 7.34%

Notes and Thoughts:

In both the 90% and 50% stock categories, the portfolio with the highest returns also had the highest deviation: more risk more reward. Simple Vanguard 3-fund portfolios were beaten by each robo advisor at both 90% and 50% stocks, but the 3-fund portfolios had lower deviations. Since the robo advisors overweight more volatile categories (e.g. high yield bonds, emerging markets, small caps), they are really more like 93% or 53% Vanguard 3-fund portfolios, in terms of betas. All in all, I’m not buying their crap about how owning 10 ETFs is superior to owning 3. The numbers don’t lie.

But that doesn’t mean I’m choosing to avoid robo advisors. Compared with Vanguard, they offer superior liquidity, the ability to easily set and manage multiple different asset allocations and goals, fractional shares. This is extremely helpful if you want to set short and mid-term goals like saving up for a car, house, wedding, etc. You can do so without manually rebalancing frequently and recalculating your overall asset allocation. It’s also easier to get money in and out of robo advisors compared with Vanguard, so I put part of my safety net to work in a 60% bonds account.

So, which robo advisor did I choose? Wisebanyan. Betterment’s 0.25% advisory fee is simply not worth it in my opinion. Wisebanyan does it for free and gives you basically the same thing. Tax loss harvesting doesn’t really apply to me because I own some of these funds in my Vanguard account so harvesting might inadvertently cause a wash sale. Regarding Wealthfront, I just fundamentally dislike the fact that they include a natural resources ETF in their portfolio. I don’t get it and I don’t like it. I’ve read their reasoning and I completely disagree—why not invest in some other low correlation asset class that is less cyclical and volatile, such as Utilities? I actually had to exclude it from my backtest because it was unavailable on the website I used. Instead, I allocated this percentage to total US stocks. They also charge 0.25% on assets over $15,000, while Wisebanyan is completely free.

If you are considering a robo advisor and you end up picking Wisebanyan, please use my referral link and we will both receive $20 for free.

https://wisebanyan.com/r/OTWx9mUnZ

COEE

  • Pencil Stache
  • ****
  • Posts: 611
Re: Backtesting Robo Advisor Portfolios
« Reply #1 on: July 14, 2017, 09:26:55 AM »
1 post?  All of this 'history'?  Referral link to a company I've never heard of before.... hmmmm... smells fishy to me!

coplar

  • 5 O'Clock Shadow
  • *
  • Posts: 11
Re: Backtesting Robo Advisor Portfolios
« Reply #2 on: July 14, 2017, 09:50:39 AM »
1 post?  All of this 'history'?  Referral link to a company I've never heard of before.... hmmmm... smells fishy to me!

1. What does it matter that I am a new member? Do you have anything valuable to contribute to the topic? Do you have anything to say about the numbers?

2. MMM has mentioned WiseBanyan himself: http://www.mrmoneymustache.com/2017/02/01/betterment-cranks-up-features-and-costs-is-it-still-worthwhile/
"I will make a point of reviewing the other robo-advisers and competing services from Wealthfront, Schwab, Vanguard, and WiseBanyan in order for this blog to be less Betterment-specific."

3. Robo advisors are a common topic of discussion here and on other investment forums. Numerous threads here have discussed them and WiseBanyan specifically (e.g. https://forum.mrmoneymustache.com/investor-alley/wise-banyan-users-thread-(wisebanyan)/ )

4. My post was in no way a ringing endorsement. I basically say this: "Robo advisors are essentially the same as a 3-fund Vanguard portfolio, but I chose to use one anyway because I like a couple of the features. Here's the one I picked, largely because it's free."
« Last Edit: July 14, 2017, 10:05:48 AM by coplar »

COEE

  • Pencil Stache
  • ****
  • Posts: 611
Re: Backtesting Robo Advisor Portfolios
« Reply #3 on: July 14, 2017, 11:42:57 AM »
1 post?  All of this 'history'?  Referral link to a company I've never heard of before.... hmmmm... smells fishy to me!

1. What does it matter that I am a new member? Do you have anything valuable to contribute to the topic? Do you have anything to say about the numbers?

There have been many times that a new member posts a first post just to advertise themselves or a company or something - not just on this site, but others I frequent as well.  I'm just pointing out that it's probably not an unbiased opinion - although I could be wrong.  But your post also tends to center around your referral link based on numbers you seemingly pulled out of the sky.

As a side note, I personally hate that MMM allows referral links on this website, but hey I don't run the website.  Referral links alone creates bias.

For valuable input - sure, I have lots -
  • Are the dividends reinvested?  And are the dividends considered in your returns?
  • Why do you think the robo investors are beating the market?  This is incredibly hard to do - and apparently the robots do it better than people?
  • What is your method for 'backtesting'?
  • Did you dollar cost average your investments during backtesting?
  • What website did you use to do your backtesting?
  • The Vanguard 3-fund at 90% and 50% equities, what is distribution of the other two funds in your backtesting and what were the distributions of the robo investors?
  • I'm assuming you're using the Bogleheads 3-fund portfolio?
  • I could go on...

coplar

  • 5 O'Clock Shadow
  • *
  • Posts: 11
Re: Backtesting Robo Advisor Portfolios
« Reply #4 on: July 14, 2017, 12:51:33 PM »
1 post?  All of this 'history'?  Referral link to a company I've never heard of before.... hmmmm... smells fishy to me!

1. What does it matter that I am a new member? Do you have anything valuable to contribute to the topic? Do you have anything to say about the numbers?

There have been many times that a new member posts a first post just to advertise themselves or a company or something - not just on this site, but others I frequent as well.  I'm just pointing out that it's probably not an unbiased opinion - although I could be wrong.  But your post also tends to center around your referral link based on numbers you seemingly pulled out of the sky.

As a side note, I personally hate that MMM allows referral links on this website, but hey I don't run the website.  Referral links alone creates bias.

For valuable input - sure, I have lots -
  • Are the dividends reinvested?  And are the dividends considered in your returns?
  • Why do you think the robo investors are beating the market?  This is incredibly hard to do - and apparently the robots do it better than people?
  • What is your method for 'backtesting'?
  • Did you dollar cost average your investments during backtesting?
  • What website did you use to do your backtesting?
  • The Vanguard 3-fund at 90% and 50% equities, what is distribution of the other two funds in your backtesting and what were the distributions of the robo investors?
  • I'm assuming you're using the Bogleheads 3-fund portfolio?
  • I could go on...
1. Yes the dividends are all reinvested and considered in the calculations.
2. I don't think they are beating the market. In fact, I said, "They all perform roughly the same as a Vanguard 3-fund portfolio matching their asset allocation," and "Since the robo advisors overweight more volatile categories... they are really more like 93% or 53% Vanguard 3-fund portfolios, in terms of betas."
3. All calculations were done on PortfolioVisualizer's asset allocation backtesting model.
4. The backtesting that I did started with an initial investment of $10,000 and no ongoing investments.
5. see 3
6. Vanguard 3-fund at 90% stocks: 63% VTI, 27% VXUS, 10% BND. Vanguard 3-fund at 50% stocks: 35% VTI, 15% VXUS, 50% BND.

For Betterment and Wisebanyan, I used the actual portfolios that they created for me during signup. For Wealthfront, I just read the portfolio section of their whitepaper. Here are the 90% stock versions of each:
Betterment - 16.20% US Stock Market 16.00% US Large Cap Value 5.20% US Mid Cap Value 4.50% US Small Cap Value 37.50% Global ex-US Stock Market 10.50% Emerging Markets 3.20% Total US Bond Market 5.20% Global Bonds (Unhedged) 1.70% Corporate Bonds
Wisebanyan - 54.00% US Stock Market 29.30% Intl Developed ex-US Market 6.80% Emerging Markets 2.60% Intermediate Term Treasury 2.10% TIPS 0.20% Short-Term Investment Grade 2.90% Corporate Bonds 1.40% High Yield Corporate Bonds 0.70% REIT

7. Yes, pretty much. There are a few different versions of this floating around (eg 40% vs 30% international). I went with 30% international.

AdrianC

  • Handlebar Stache
  • *****
  • Posts: 1211
  • Location: Cincinnati
Re: Backtesting Robo Advisor Portfolios
« Reply #5 on: July 14, 2017, 01:45:13 PM »
Nice work.

Betterment's fee used to be lower (0.1%?), until they needed to make some money.

I'd be concerned about Wisebanyan's business model. Might they also be asking for a management fee at some point? Their AUM is also low ($100M?).

But that doesn’t mean I’m choosing to avoid robo advisors. Compared with Vanguard, they offer superior liquidity, the ability to easily set and manage multiple different asset allocations and goals, fractional shares. This is extremely helpful if you want to set short and mid-term goals like saving up for a car, house, wedding, etc. You can do so without manually rebalancing frequently and recalculating your overall asset allocation. It’s also easier to get money in and out of robo advisors compared with Vanguard, so I put part of my safety net to work in a 60% bonds account.
Superior liquidity? How so?

Which ETFs are Wisebanyan using?

Is a Vanguard Lifestrategy fund just as easy? And have you could own different Lifestrategy funds for different investment goals.

Radagast

  • Magnum Stache
  • ******
  • Posts: 2541
  • One Does Not Simply Work Into Mordor
Re: Backtesting Robo Advisor Portfolios
« Reply #6 on: July 14, 2017, 01:59:48 PM »
Why do you think the robo investors are beating the market?  This is incredibly hard to do - and apparently the robots do it better than people?
Almost everyone can beat the market, no problem. It starts becoming hard to beat the market as you check more of the following boxes:
X After accounting for costs, expenses, market impacts, etc.
X On a risk adjusted basis
X Consistently
X Over long periods of time
If you check all of them almost nobody can beat the market. It seems like robo-advisors do not check the "risk adjusted basis" box. They push you to take more risk, while the robo-advisor gets a guarantee of more return. You get more risk and maybe more return, or maybe not.

My biggest critique is that the robo-advisors all seem to use much more international than your three fund comparison, which would throw off the results. My other concern is that your back testing seems to be using generic funds rather than the specific funds robo-advisors use. Probably the funds they actually use have not been around that long, though. There is probably a lot of false precision in the results.

coplar

  • 5 O'Clock Shadow
  • *
  • Posts: 11
Re: Backtesting Robo Advisor Portfolios
« Reply #7 on: July 14, 2017, 03:03:31 PM »
Nice work.

Betterment's fee used to be lower (0.1%?), until they needed to make some money.

I'd be concerned about Wisebanyan's business model. Might they also be asking for a management fee at some point? Their AUM is also low ($100M?).

But that doesn’t mean I’m choosing to avoid robo advisors. Compared with Vanguard, they offer superior liquidity, the ability to easily set and manage multiple different asset allocations and goals, fractional shares. This is extremely helpful if you want to set short and mid-term goals like saving up for a car, house, wedding, etc. You can do so without manually rebalancing frequently and recalculating your overall asset allocation. It’s also easier to get money in and out of robo advisors compared with Vanguard, so I put part of my safety net to work in a 60% bonds account.
Superior liquidity? How so?

Which ETFs are Wisebanyan using?

Is a Vanguard Lifestrategy fund just as easy? And have you could own different Lifestrategy funds for different investment goals.

You're definitely right about their business model. Hopefully they end up doing well with a free model like Robinhood has.

I believe the robo advisors have better liquidity because it is easier to buy/sell and the funds clear faster.

Their portfolio consists mainly of vanguard ETFs. They are all index funds and the average expense ratio is 0.12%. I gave the exact breakdown in another reply above.

The vanguard life strategy funds are indeed very similar to the robo advisor approach. I hope vanguard offers them as ETFs in the future, but unfortunately the mutual funds all have minimums in the thousands. If vanguard does do these in etf format I will definitely switch.

coplar

  • 5 O'Clock Shadow
  • *
  • Posts: 11
Re: Backtesting Robo Advisor Portfolios
« Reply #8 on: July 14, 2017, 03:09:10 PM »
Why do you think the robo investors are beating the market?  This is incredibly hard to do - and apparently the robots do it better than people?
Almost everyone can beat the market, no problem. It starts becoming hard to beat the market as you check more of the following boxes:
X After accounting for costs, expenses, market impacts, etc.
X On a risk adjusted basis
X Consistently
X Over long periods of time
If you check all of them almost nobody can beat the market. It seems like robo-advisors do not check the "risk adjusted basis" box. They push you to take more risk, while the robo-advisor gets a guarantee of more return. You get more risk and maybe more return, or maybe not.

My biggest critique is that the robo-advisors all seem to use much more international than your three fund comparison, which would throw off the results. My other concern is that your back testing seems to be using generic funds rather than the specific funds robo-advisors use. Probably the funds they actually use have not been around that long, though. There is probably a lot of false precision in the results.
I agree with pretty much everything you wrote. The deviations of the robo advisors are indeed higher than the bogleheads portfolio. But not by much, usually less than a percent.

Regarding back testing generic indices vs specific funds, I don't believe this is a significant source of error in the model. For every asset class all of the robo advisors use index funds from blackrock or vanguard, all of which track their indices remarkably well and do so with very low expenses. So while I agree this is a source of error I believe that error to be negligible.

AdrianC

  • Handlebar Stache
  • *****
  • Posts: 1211
  • Location: Cincinnati
Re: Backtesting Robo Advisor Portfolios
« Reply #9 on: July 15, 2017, 10:42:11 AM »
I believe the robo advisors have better liquidity because it is easier to buy/sell and the funds clear faster.
It's easy to buy and sell at Vanguard too. I've never pulled money out of Vanguard and don't know how long it takes. Can't imagine it's much slower than anywhere else.

Quote
The vanguard life strategy funds are indeed very similar to the robo advisor approach. I hope vanguard offers them as ETFs in the future, but unfortunately the mutual funds all have minimums in the thousands. If vanguard does do these in etf format I will definitely switch.

Yes, $3K minimum on Lifestrategy.

Vanguard does have Total World Stock ETF (VT) 53% US, 38% Developed International, 9% Emerging Markets. It's a good option also. I have my kids money in ETFs with a similar allocation. I should have just used VT. They'll get whacked with cap gains if I switched them now.

Mighty-Dollar

  • Bristles
  • ***
  • Posts: 422
Re: Backtesting Robo Advisor Portfolios
« Reply #10 on: July 16, 2017, 04:01:28 AM »
It's all about bond/stock allocation ratio. Just pick a total bond market index fund and a total stock market index fund. You'll very closely replicate any robo advisor that recommends a mid cap growth, mid cap value, small cap growth, small cap value, large cap value, large cap growth, short/mid and long term bonds. Six or half a dozen. Same thing. Cut out the complication. Keep it simple. Lower trading costs.

COEE

  • Pencil Stache
  • ****
  • Posts: 611
Re: Backtesting Robo Advisor Portfolios
« Reply #11 on: July 16, 2017, 09:30:31 AM »
So I took a few minutes to 'sniff-test' your conclusions.

I used the following website: http://stockcharts.com/freecharts/perf.php?VTI,VBTLX,SWISX to look at similar funds (that actually existed in 2004) to see what the gains would be on a $10k initial investment.  PerfCharts uses dividend reinvestment and fee structure to calculate want your [unrealized] gains would actually be on a one-time investment.  It does not take into account taxes, since that's somewhat personal.

My allocation:
           90% stock      50% stock
VTI          63%                35%
SWISX    27%                15%
VBTLX    10%                50%

Time period Dec 31, 2002 to July 14th, 2017

I'm calculating a 15.02%/yearly for the 90% and 10.77%/year for the 50% scenarios.  So the performance was actually much better than what you report.  Although the funds are probably not exactly the same as you report, even if I calculate a 0% gain in both the bond and international fund (leaving just the VTI, which you used), 12.44% and 6.91%.  So the 90% is still better than what you report (by 3 percentage points!), and the 50% doesn't do as good because of the bond rich investment.

For comparison, can you provide the exact funds that were invested using Wisebanyan?  This will provide us an idea of the expense ratios, and how this effects performance.  You said you had "Wisebanyan - 54.00% US Stock Market 29.30% Intl Developed ex-US Market 6.80% Emerging Markets 2.60% Intermediate Term Treasury 2.10% TIPS 0.20% Short-Term Investment Grade 2.90% Corporate Bonds 1.40% High Yield Corporate Bonds 0.70% REIT".

See attached file for my quick swags.

Finally, how do you backtest funds that weren't even in existence in 2003?  I'm not even aware of any ExUS funds that were around back then, but I didn't look real hard.

StudentEngineer

  • 5 O'Clock Shadow
  • *
  • Posts: 88
Re: Backtesting Robo Advisor Portfolios
« Reply #12 on: July 16, 2017, 02:04:03 PM »
Commenting to follow.  Good discussion!

coplar

  • 5 O'Clock Shadow
  • *
  • Posts: 11
Re: Backtesting Robo Advisor Portfolios
« Reply #13 on: July 17, 2017, 07:35:31 AM »
So I took a few minutes to 'sniff-test' your conclusions.

I used the following website: http://stockcharts.com/freecharts/perf.php?VTI,VBTLX,SWISX to look at similar funds (that actually existed in 2004) to see what the gains would be on a $10k initial investment.  PerfCharts uses dividend reinvestment and fee structure to calculate want your [unrealized] gains would actually be on a one-time investment.  It does not take into account taxes, since that's somewhat personal.

My allocation:
           90% stock      50% stock
VTI          63%                35%
SWISX    27%                15%
VBTLX    10%                50%

Time period Dec 31, 2002 to July 14th, 2017

I'm calculating a 15.02%/yearly for the 90% and 10.77%/year for the 50% scenarios.  So the performance was actually much better than what you report.  Although the funds are probably not exactly the same as you report, even if I calculate a 0% gain in both the bond and international fund (leaving just the VTI, which you used), 12.44% and 6.91%.  So the 90% is still better than what you report (by 3 percentage points!), and the 50% doesn't do as good because of the bond rich investment.

For comparison, can you provide the exact funds that were invested using Wisebanyan?  This will provide us an idea of the expense ratios, and how this effects performance.  You said you had "Wisebanyan - 54.00% US Stock Market 29.30% Intl Developed ex-US Market 6.80% Emerging Markets 2.60% Intermediate Term Treasury 2.10% TIPS 0.20% Short-Term Investment Grade 2.90% Corporate Bonds 1.40% High Yield Corporate Bonds 0.70% REIT".

See attached file for my quick swags.

Finally, how do you backtest funds that weren't even in existence in 2003?  I'm not even aware of any ExUS funds that were around back then, but I didn't look real hard.

1. The reason we come up with different numbers is because mine is inflation-adjusted CAGR while you calculated non-inflation-adjusted AAGR.

2. The ETFS they use are:
Total US Market - VTI
International Developed: VEA
Emerging: VWO
Investment Grade: LQD
Short Term Corporate: VCSH
High yield corporate: SJNK
Treasuries: VGIT
inflation protected: TIP
REITS: VNQ
Avg. expense ratio: 0.12%

3. I backtested the asset allocations, not the exact ETFs. Honestly I don't understand the supposed importance of this. All broadly diversified index funds are extremely good at tracking their indices, so backtesting the index seems to be a solid method.
« Last Edit: July 17, 2017, 07:37:23 AM by coplar »

 

Wow, a phone plan for fifteen bucks!