Differences between retirement calculators-FI Calc, cFIREsim, Rich Broke Dead

[BeanCounter]

What causes the difference between success rates in various retirement calculators?

I have attempted to put the same information into all three calculators- FI Calc, cFIREsim and Rich, Broke or Dead and I'm getting different success rates.
Same time period, asset allocation, constant withdrawal.

Example-
4% withdrawal, adjusted for inflation for 50 years. Allocation of 80% stock, 15% bonds, 5% cash. (do not want to debate if any of this is right just the differences of the simulation calculator output)

FI Calc- 88.3% success
cFIREsim- 74.5% success
Rich Broke or dead- 79% success

I would have thought they'd be closer than that. They are using the same data sets right?

[Ron Scott]

My guess is that’s par for the course. These products need to make so many assumptions.

Just pick two economic constraints for example. Inflation: how do you model inflation overtime? How do you model the impact that inflation will have in the future on asset returns? Taxes: How to model changes in the tax rate structure or tax law regarding investment returns, etc.? How to determine the impact taxes will have on asset returns during a multi-decade retirement period?

This is economics, which is social science, which is difficult. This leads experts to disagree. Enjoy.

[BeanCounter]

My guess is that’s par for the course. These products need to make so many assumptions.

Just pick two economic constraints for example. Inflation: how do you model inflation overtime? How do you model the impact that inflation will have in the future on asset returns? Taxes: How to model changes in the tax rate structure or tax law regarding investment returns, etc.? How to determine the impact taxes will have on asset returns during a multi-decade retirement period?

This is economics, which is social science, which is difficult. This leads experts to disagree. Enjoy.

Have you used either of those three Monte Carlo simulators? They are not making assumptions about inflation or return rates. They have 100 years of historical data (rate of return for each year, inflation rate) and they run the assumptions I select against those historical actuals to come up with a success rate for the period of time you select. So I would think the data sets should be all the same. The only thing I can come up with that could be different is the timing of applying the draw or inflation to the historical balance. But I haven’t dug through the cab files to figure it out.

[flyingaway]

By your examples, they did not differ very much. None of the calculators is based on science, they are based on assumptions and opinions. So some differences in percentage are expected. How to use and explain the data is up to you.

[cannotWAIT]

I think this is an interesting question and one that the developer of these simulators ought to be able to answer. You should reach out to them and report back!

[BlueHouse]

Could it be the tax assumptions?  cFiresim doesn't have you to enter your expected tax rate, so it's calculating it based on your inputs of spending as income.  Rich, Dead, or Broke has you enter in an Average Tax rate over 30+ years of retirement.  Many of us try to minimize our tax burden, so I'm sure I'm entering an Average tax that is not quite right. 

If you enter your details into cFiresim, using investigative options to calculate the "Maximum Initial Yearly Spending For 100% Success
Minimum Success Rate", then enter the value returned into Rich, Dead, Broke with a 0% tax rate, your results should be under .5% different.




 

[BeanCounter]

Could it be the tax assumptions?  cFiresim doesn't have you to enter your expected tax rate, so it's calculating it based on your inputs of spending as income.  Rich, Dead, or Broke has you enter in an Average Tax rate over 30+ years of retirement.  Many of us try to minimize our tax burden, so I'm sure I'm entering an Average tax that is not quite right. 

If you enter your details into cFiresim, using investigative options to calculate the "Maximum Initial Yearly Spending For 100% Success
Minimum Success Rate", then enter the value returned into Rich, Dead, Broke with a 0% tax rate, your results should be under .5% different.

Huh. I don’t think any of those simulation calculators should be including tax assumptions. That would be impossible to model. I’ve always assumed an amount that I anticipate to pay for taxes and include that in my withdrawal amount. You have to figure out the likelihood of being able to withdraw that amount successfully and then you pay the tax!
Also none of the CSV files from these calculators show tax.
But you are correct that 100% success is achieved with a 3% draw rate (for 50 years) across all calculators.

[Beach_Bound]

Interesting. Yes, according to their documentation, they all use the same data source for stock returns, bond returns, and inflation (http://www.econ.yale.edu/~shiller/data.htm), so that shouldn't be a source of the discrepancy.

cFIREsim defaults to fees of 0.18% and growth of cash of 0.25%. If you change both of those to zero, the success rate increases from 74.5% to 79.4%.

Rich, Broke, or Dead defaults to investment fees of 0.3%. If you change that to zero, the success rate increases from 81% to 84%.

That reduces the spread from 74.5-88.3% to 79.4-88.3%, which is still larger than I would have expected. The remaining difference between the calculators is likely due to order in which returns, inflation, withdrawals, and rebalancing are applied. Understanding it further may require digging into the csv output files.

[BeanCounter]

Interesting. Yes, according to their documentation, they all use the same data source for stock returns, bond returns, and inflation (http://www.econ.yale.edu/~shiller/data.htm), so that shouldn't be a source of the discrepancy.

cFIREsim defaults to fees of 0.18% and growth of cash of 0.25%. If you change both of those to zero, the success rate increases from 74.5% to 79.4%.

Rich, Broke, or Dead defaults to investment fees of 0.3%. If you change that to zero, the success rate increases from 81% to 84%.

That reduces the spread from 74.5-88.3% to 79.4-88.3%, which is still larger than I would have expected. The remaining difference between the calculators is likely due to order in which returns, inflation, withdrawals, and rebalancing are applied. Understanding it further may require digging into the csv output files.

Yep! I think you nailed it. I never noticed those defaults before.
And now I see the tax area in rich, broke or dead. That makes no sense to me as there are way too many factors how tax is applied to draw down.

[moof]

Curious.  My own case differs slightly, 94.3% vs 91.9%.  I'll have to look into it more by dragging the CSV outputs into Excel and comparing.  One really important detail to watch for is what is assumed around the starting/ending boundaries.  With the granularity of 1 year it make a big difference as to how the author applied the gains and withdrawals (end of year, vs. beginning for example), and withdrawals are likely spread over a year and not always front-loaded in January.  I've always wanted something that let me specify to the nearest month and just prorated accordingly.  Not all of us want to retire evenly on an annual boundary after all.  I mean a lot of things naturally are mid year events like social security eligibility, college costs, etc.  Annual lump sum withdrawals in January vs. December has a sizeable impact on a SWR that all these annual based calculators cannot account for.

[Ron Scott]

Interesting. Yes, according to their documentation, they all use the same data source for stock returns, bond returns, and inflation (http://www.econ.yale.edu/~shiller/data.htm), so that shouldn't be a source of the discrepancy.

Maybe someone who actually trusts these calculators and data to determine the probability of running out of money in retirement can tell us about modeling economic variables. I’ve played with these things for fun but I don’t “use” them for anything and admit I don’t understand how they work.

Take inflation for example:
Do the models account for the effects that stock/bond returns have on inflation and vice versa?
Can the stats push the models into periods of something like stagflation? Recession?
I assume the federal reserve has learned something over time about tackling inflation. Is this reflected in the models?

[BeanCounter]

Interesting. Yes, according to their documentation, they all use the same data source for stock returns, bond returns, and inflation (http://www.econ.yale.edu/~shiller/data.htm), so that shouldn't be a source of the discrepancy.

Maybe someone who actually trusts these calculators and data to determine the probability of running out of money in retirement can tell us about modeling economic variables. I’ve played with these things for fun but I don’t “use” them for anything and admit I don’t understand how they work.

Take inflation for example:
Do the models account for the effects that stock/bond returns have on inflation and vice versa?
Can the stats push the models into periods of something like stagflation? Recession?
I assume the federal reserve has learned something over time about tackling inflation. Is this reflected in the models?

These three calculators we are discussing are really monte carlo simulators and are not the same as "retirement calculators" that are put out by various websites. All the simulators do is take historical data and apply the information you input to the historical data. There aren't many assumptions built in. This is why I was surprised they didn't match. I just didn't notice that there was a default on the input cell for fees.

Anyway when you say "trust" these calculators, I don't think there is anything really to trust. Other than the assumption that some version of future performance could mirror some version of past. I spent 20 years in corporate finance and when we are modeling budgets and future performance of business lines this is what we do. We use past data to model possible future outcomes. That's all these simulators are doing.

So if I tell cfiresim for example, that I have a $1M portfolio and I want to draw $30k off of it for 50 years, I input that data and the investment mix of my portfolio and it models it against 100 years of historical performance for equities, bonds and inflation data. And it does it for 100 50 year cycles.
You can spit out the CSV data and see EXACTLY what the simulator is doing.
The only thing you can't see is the impacts of the assumptions you are putting in on fees and the timing of the draws. I can't seem to figure out how that is being applied, but that's pretty immaterial.  The simplicity of these three simulators is what I like about them. I have also exported the CSV files and used the historical data to do my own math too.

[BeanCounter]

Curious.  My own case differs slightly, 94.3% vs 91.9%.  I'll have to look into it more by dragging the CSV outputs into Excel and comparing.  One really important detail to watch for is what is assumed around the starting/ending boundaries.  With the granularity of 1 year it make a big difference as to how the author applied the gains and withdrawals (end of year, vs. beginning for example), and withdrawals are likely spread over a year and not always front-loaded in January.  I've always wanted something that let me specify to the nearest month and just prorated accordingly.  Not all of us want to retire evenly on an annual boundary after all.  I mean a lot of things naturally are mid year events like social security eligibility, college costs, etc.  Annual lump sum withdrawals in January vs. December has a sizeable impact on a SWR that all these annual based calculators cannot account for.

I pulled down all three CSVs the other day and it looked like cFIRESim is the only one that is including 2022 data. (I didn't spend too much time on it though). I think that could be why it's results are lower.

[Ron Scott]

Interesting. Yes, according to their documentation, they all use the same data source for stock returns, bond returns, and inflation (http://www.econ.yale.edu/~shiller/data.htm), so that shouldn't be a source of the discrepancy.

Maybe someone who actually trusts these calculators and data to determine the probability of running out of money in retirement can tell us about modeling economic variables. I’ve played with these things for fun but I don’t “use” them for anything and admit I don’t understand how they work.

Take inflation for example:
Do the models account for the effects that stock/bond returns have on inflation and vice versa?
Can the stats push the models into periods of something like stagflation? Recession?
I assume the federal reserve has learned something over time about tackling inflation. Is this reflected in the models?

These three calculators we are discussing are really monte carlo simulators and are not the same as "retirement calculators" that are put out by various websites. All the simulators do is take historical data and apply the information you input to the historical data. There aren't many assumptions built in. This is why I was surprised they didn't match. I just didn't notice that there was a default on the input cell for fees.

Anyway when you say "trust" these calculators, I don't think there is anything really to trust. Other than the assumption that some version of future performance could mirror some version of the past

Yes, that sounds about right. Simple models of complex phenomena can react significantly to even the slightest changes in assumptions, which is what I suspect we’re seeing here. Buyer beware…