Author Topic: cfiresim acting up  (Read 10979 times)

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
cfiresim acting up
« on: November 17, 2015, 12:48:17 PM »
So I've been spending lots of quality time on cfiresim (rather than working) and while I really like the tool, it seems a bit buggy.  A number of times I've changed numbers that should decrease the size of my assets in the future, but when I run the simulations, my assets are larger.  Also, it isn't uncommon to have the simulations just go literally off the charts in ways that are completely inconsistent with the inputs I have down.

This is happening enough to make me wonder about the soundness of the outputs when it appears to be acting 'normal'.

Any insights or words of encouragement?

StockBeard

  • Pencil Stache
  • ****
  • Posts: 643
  • Age: 38
    • How To Retire Early?
Re: cfiresim acting up
« Reply #1 on: November 17, 2015, 01:02:33 PM »
cFireSim has a hard limit to 10 simulations it can run for logged-out users. After that, you'll always get the results from the 10th one, until you force reload the page.
Is that what you're seeing?

bo_knows

  • Pencil Stache
  • ****
  • Posts: 819
  • Age: 39
  • Location: Fairfax, VA, USA
    • The Crowdsourced FIRE simulator
Re: cfiresim acting up
« Reply #2 on: November 17, 2015, 01:18:29 PM »
You don't give enough specifics. Sign up on cFIREsim's forums, and post the specific scenarios in which you see problems.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #3 on: November 18, 2015, 08:42:54 AM »
cFireSim has a hard limit to 10 simulations it can run for logged-out users. After that, you'll always get the results from the 10th one, until you force reload the page.
Is that what you're seeing?

That does explain a number of of the things I've seen ("hey, I keep increasing my expenditures and I'm still good!")...but...I'm logged in when doing this

Also, I have changed the spending floor or ceiling or z-score under a variable spending model and a few times it  has gone haywire...e.g. decreasing my max spending and the simulation removes the max completely...changing the z from .5 to .7 even with a narrow floor and ceiling range and the model runs negative after a few years for all scenarios when before all were very much above zero for all scenarios
« Last Edit: November 18, 2015, 08:47:34 AM by spokey doke »

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #4 on: November 18, 2015, 08:59:30 AM »
You don't give enough specifics. Sign up on cFIREsim's forums, and post the specific scenarios in which you see problems.

I have signed up, but the only forums I looked at had no apparent traffic for some time (but now I see the bugs forum does have a few recent entries).
« Last Edit: November 18, 2015, 05:52:00 PM by spokey doke »

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #5 on: November 18, 2015, 05:51:20 PM »
I just ran the same simulation in the new and the old sites, and the new one give 100% success and the old 95%...I even logged out of the new site, refreshed, logged in, then re-entered all the numbers, and did the same for the old site...same numbers, different outcomes

And the results from the new site of 100% seems sketchy, given that I have a pretty high spending rate entered, and substantial periodic additional expenses plugged in...seemed too good to be true
« Last Edit: November 18, 2015, 05:55:16 PM by spokey doke »

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #6 on: November 20, 2015, 04:37:28 AM »
You're giving zero info here.

What numbers did you put in?  If it's reproducible, tell us the exact inputs so BK can reproduce, and fix. (And put those numbers on the cFIREsim forum).

He will fix bugs, and is happy to have them pointed out, but you have to give the info so he knows what you're talking about. :)
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #7 on: November 22, 2015, 08:32:00 AM »
You're giving zero info here.

What numbers did you put in?  If it's reproducible, tell us the exact inputs so BK can reproduce, and fix. (And put those numbers on the cFIREsim forum).

He will fix bugs, and is happy to have them pointed out, but you have to give the info so he knows what you're talking about. :)

OK then...the attached are screenshots and I can only attach 4 at a time, so I make another post below.

Mind you, these numbers reflect a quite a few rounds of increasing annual spending, adding periodic spending, and still getting 100% success rates (these numbers returned 95% success from the older FireCalc)...we are not so mustachian at this point...

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #8 on: November 22, 2015, 08:32:31 AM »
and the last one...

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #9 on: November 22, 2015, 10:43:33 AM »
And what is your issue?  80k spending on 1.86MM is just above 4% SWR, which is 95% safe historically.  Then you add in the fact that you're willing to cut your spending in half (variable spending with 40k floor) and only increase it 20% (95k ceiling) and adding a pension and social security, even with the extra 20k one time expenses every 5 years, you're totally fine (cause cFIREsim can always cut your normal spending that year if you're in trouble).

I would absolutely expect a 100% success rate in this scenario.

So what's the problem?

You mentioned earlier "A number of times I've changed numbers that should decrease the size of my assets in the future, but when I run the simulations, my assets are larger."

Okay. What numbers did you change and what result did you get?

No need for more screenshots, unless that's easier, you can just type "I changed X assumption to Y, and that increased/decreased my assets or success rate from this to this" type thing. :)
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #10 on: November 22, 2015, 02:04:42 PM »
And what is your issue?  80k spending on 1.86MM is just above 4% SWR, which is 95% safe historically.  Then you add in the fact that you're willing to cut your spending in half (variable spending with 40k floor) and only increase it 20% (95k ceiling) and adding a pension and social security, even with the extra 20k one time expenses every 5 years, you're totally fine (cause cFIREsim can always cut your normal spending that year if you're in trouble).

I would absolutely expect a 100% success rate in this scenario.

So what's the problem?

You mentioned earlier "A number of times I've changed numbers that should decrease the size of my assets in the future, but when I run the simulations, my assets are larger."

Okay. What numbers did you change and what result did you get?

No need for more screenshots, unless that's easier, you can just type "I changed X assumption to Y, and that increased/decreased my assets or success rate from this to this" type thing. :)

Thanks for hanging with me while I flail around a bit...

One issue is that FireCalc gives 95% success rate with the same numbers

Another is that I changed the z score under the variable spending model from .5 to .7 and ALL results in the simulation went negative within a fairly short period of time. As I understand what the z-score does, that seems counter intuitive, particularly since my floor spending value was much lower than my initial spending rate than my ceiling value was above it (more room to decrease spending in bad times than room to increase in good times). 

Another time I increased the annual spending by 5-10K/year and my lowest balances in the simulation were greater than with a lower annual spending rate. 

So while overall things seem in the ballpark most of the time, the number of counter-intuitive and quirky results that have popped up along the way have shaken my confidence...AND/OR, I don't understand as well as I thought..AND/OR I'm still not willing to let myself believe the math because that would mean taking the plunge pretty much any time now
« Last Edit: November 22, 2015, 02:07:45 PM by spokey doke »

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #11 on: November 22, 2015, 02:13:14 PM »
FIRECalc has a known flaw in how it calculates bond returns, so that's generally the difference in results.

But I didn't think FIRECalc even HAD a variable spending model with ceilings and floors, so how are you even comparing them?

Also start with the above scenario screenshotted and the exact change made (in z score or annual spending, or whatever). Your answers are still vague.  Not what you did in the past, go do it now and find an actual thing that you think is incorrect, otherwise without knowing the scenario it's hard to duplicate to look for bugs. :)
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #12 on: November 22, 2015, 03:39:05 PM »

But I didn't think FIRECalc even HAD a variable spending model with ceilings and floors, so how are you even comparing them?


my mistake, I meant the old cfiresim site (which does have the variable spending rate).

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #13 on: November 22, 2015, 09:23:23 PM »

Also start with the above scenario screenshotted and the exact change made (in z score or annual spending, or whatever). Your answers are still vague.  Not what you did in the past, go do it now and find an actual thing that you think is incorrect, otherwise without knowing the scenario it's hard to duplicate to look for bugs. :)

Sorry, my experience, for which I don't have perfect recall or records, suggests some questionable results.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
cfiresim acting up
« Reply #14 on: November 23, 2015, 01:50:57 AM »
Right, I'm not asking you to recall, just duplicate.  Go play around and see if you can find dodgy results. Cause I see no problems with your scenario.  But put it in, tweak things, and if you see something weird, write down what you tweaked and ask about it here.

Otherwise I'd be more likely to ascribe it to minor confusion at the time.  But hopefully you find some bugs in the new version, help out the dev, and get yourself some more accurate results. :)
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #15 on: November 25, 2015, 03:23:06 PM »
My biggest concern right now is that I can get 100% success on the new cfiresim site, and with the same numbers I now get 83% success on the old cfiresim site.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #16 on: November 25, 2015, 03:49:23 PM »

My biggest concern right now is that I can get 100% success on the new cfiresim site, and with the same numbers I now get 83% success on the old cfiresim site.

With the above screenshotted numbers, or some other scenario?

That is a big problem.  I would trust the old site over the newer, it's more tested (and trusting a lower number is the safer way to go anyways).
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #17 on: November 26, 2015, 08:27:03 AM »

My biggest concern right now is that I can get 100% success on the new cfiresim site, and with the same numbers I now get 83% success on the old cfiresim site.

With the above screenshotted numbers, or some other scenario?

That is a big problem.  I would trust the old site over the newer, it's more tested (and trusting a lower number is the safer way to go anyways).

With very similar numbers,...and with the new site, I kept cranking up the annual spending until 110K/yr, with another $25K periodic spending every 5 years before getting anything less than 100% success.  That makes my doubt the results (perhaps if it sounds too good to be true...)

Mmm_Donuts

  • Bristles
  • ***
  • Posts: 396
Re: cfiresim acting up
« Reply #18 on: November 27, 2015, 10:14:01 AM »
I have noticed something similar. With the old site my numbers are at a 75% success rate, with the new site I'm at 95%. I'll try again when I have time, to verify all numbers are the same, but that was my recollection.

Retire-Canada

  • Walrus Stache
  • *******
  • Posts: 7739
Re: cfiresim acting up
« Reply #19 on: November 27, 2015, 05:12:54 PM »
Running the same numbers:

- old site = 69% success
- new site = 98% success

So something is different between the two.

I'm using a $30K - $50K variable spending plan.

I reported the issue via the cFIREsim problem link.
« Last Edit: November 27, 2015, 05:54:59 PM by Retire-Canada »

matchewed

  • Magnum Stache
  • ******
  • Posts: 4381
  • Location: CT
Re: cfiresim acting up
« Reply #20 on: November 28, 2015, 07:42:22 AM »

My biggest concern right now is that I can get 100% success on the new cfiresim site, and with the same numbers I now get 83% success on the old cfiresim site.

With the above screenshotted numbers, or some other scenario?

That is a big problem.  I would trust the old site over the newer, it's more tested (and trusting a lower number is the safer way to go anyways).

With very similar numbers,...and with the new site, I kept cranking up the annual spending until 110K/yr, with another $25K periodic spending every 5 years before getting anything less than 100% success.  That makes my doubt the results (perhaps if it sounds too good to be true...)

They need to be exact same numbers, not similar. You need to compare apples to apples when looking for bugs.

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #21 on: November 28, 2015, 08:54:42 AM »

My biggest concern right now is that I can get 100% success on the new cfiresim site, and with the same numbers I now get 83% success on the old cfiresim site.

With the above screenshotted numbers, or some other scenario?

That is a big problem.  I would trust the old site over the newer, it's more tested (and trusting a lower number is the safer way to go anyways).

With very similar numbers,...and with the new site, I kept cranking up the annual spending until 110K/yr, with another $25K periodic spending every 5 years before getting anything less than 100% success.  That makes my doubt the results (perhaps if it sounds too good to be true...)

They need to be exact same numbers, not similar. You need to compare apples to apples when looking for bugs.

they were exactly the same numbers used on both sites...I was using 'similar' in reference to the numbers I had posted earlier...and so the new numbers were similar to those posted numbers, and the new numbers I was reporting on were identical on both sites, but the new site returned 100% success and the old 83%

the other observation of cranking up annual spending to 110K with 100% success was just in reference to the new site, so I was not making a comparison with that statement

thanks for others chiming in with some confirmation, the significant difference in results leaves me without a basis for evaluating or having confidence in either
« Last Edit: November 28, 2015, 08:57:55 AM by spokey doke »

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #22 on: November 28, 2015, 10:13:05 AM »
You can always fall back to (the inferior) FIRECalc and see how those numbers compare to see which is right. I'd guess the old cFIREsim.
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

bo_knows

  • Pencil Stache
  • ****
  • Posts: 819
  • Age: 39
  • Location: Fairfax, VA, USA
    • The Crowdsourced FIRE simulator
Re: cfiresim acting up
« Reply #23 on: February 18, 2016, 08:03:39 AM »
So, I sort of forgot about this thread and didn't see all of the great information posted on here.  I wanted to unearth it to gain a better understanding and maybe clarify some things.

First, I can't seem to duplicate the differences between the old cFIREsim and the new.  Using Spokey Doke's numbers, the old site is giving me 94.79% success, and the new site is giving me 100%. With the changes in the versions and how things are calculated, this is not really a surprise to me. Though, I'm not sure why I can't duplicate Spokey Doke's 83% number.

For some clarification, here is a copypasta of some of the differences in the old/new:

Quote
There are a few slight differences between the old and new, but that average ending portfolio difference seems large.

The biggest difference is how spending is taken out. In the old cFIREsim for any given year, here is how things are calculated:

- Jan 1 - Portfolio $X
- Dec 31st - Portfolio $X + Market Gains - Spending

The new version, takes spending out on January 1st (which most people seem to think is more intuative to how people would actually withdrawal) and thus that amount doesn't necessarily get the benefit of market gains. So, it's like:

- Jan 1st - Portfolio = $X - Spending
- Dec 31st - Portfolio = (Above amount) + Market gains.

This causes some interesting effects over time, which can swing results a bit.  I also did a few simple comparisons between the old/new when using Variable spending, and the results are fairly close (some ending portfolios are different by 10-20%, but to me that isn't entirely unexpected).

While this is not an excuse, please keep in mind that my experience with all of the variable spending methods has led me to this conclusion: A LOT of what you see during the long term, when using these methods, ends up being counter intuitive. It's pretty confusing.

If anyone can give more examples of this, I can try to figure out the problem.  Though, very soon I will probably completely remove the old site from production.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #24 on: February 18, 2016, 08:11:04 AM »
That makes sense--taking spending out at the beginning rather than end is more likely.

Can that be an option? Like choose between one of three options: taking spending out at beginning of year, end, or 1/12th monthly?
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

BBub

  • Pencil Stache
  • ****
  • Posts: 774
  • Age: 35
  • Location: Deep South
Re: cfiresim acting up
« Reply #25 on: February 18, 2016, 08:16:16 AM »
Quote
There are a few slight differences between the old and new, but that average ending portfolio difference seems large.

The biggest difference is how spending is taken out. In the old cFIREsim for any given year, here is how things are calculated:

- Jan 1 - Portfolio $X
- Dec 31st - Portfolio $X + Market Gains - Spending

The new version, takes spending out on January 1st (which most people seem to think is more intuative to how people would actually withdrawal) and thus that amount doesn't necessarily get the benefit of market gains. So, it's like:

- Jan 1st - Portfolio = $X - Spending
- Dec 31st - Portfolio = (Above amount) + Market gains.

All you're doing there is delaying retirement spending OMY.  Get it? 

bo_knows

  • Pencil Stache
  • ****
  • Posts: 819
  • Age: 39
  • Location: Fairfax, VA, USA
    • The Crowdsourced FIRE simulator
Re: cfiresim acting up
« Reply #26 on: February 18, 2016, 10:26:01 AM »
Quote
There are a few slight differences between the old and new, but that average ending portfolio difference seems large.

The biggest difference is how spending is taken out. In the old cFIREsim for any given year, here is how things are calculated:

- Jan 1 - Portfolio $X
- Dec 31st - Portfolio $X + Market Gains - Spending

The new version, takes spending out on January 1st (which most people seem to think is more intuative to how people would actually withdrawal) and thus that amount doesn't necessarily get the benefit of market gains. So, it's like:

- Jan 1st - Portfolio = $X - Spending
- Dec 31st - Portfolio = (Above amount) + Market gains.

All you're doing there is delaying retirement spending OMY.  Get it?

I think that the OLD method delayed spending a year, since it allowed the entire first year to accrue market gains before taking spending.

Quote
Can that be an option? Like choose between one of three options: taking spending out at beginning of year, end, or 1/12th monthly?

It might be possible to allow you to choose beginning or end, but not 1/12 monthly unless I integrate all of the monthly Shiller data (which last time I tried made it too slow to run in a browser). But, I feel like making it an option is superfluous. How many users are even going to understand the difference and why to choose one over the other?

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #27 on: February 18, 2016, 11:26:38 AM »
How many users are even going to understand the difference and why to choose one over the other?

That's probably true of most features.

I do think the beginning of year withdrawal makes way more sense, and, from what I understand from most early retirees I've heard discuss their withdrawal strategies (primarily on E-R.org), is what they actually do.

But the 1/12th option is the other one, and the one I prefer--leave your money invested as long as possible.  But if it takes 12x more computing power, and would introduce significant lag, it makes sense why it's not worth adding this feature.
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

bo_knows

  • Pencil Stache
  • ****
  • Posts: 819
  • Age: 39
  • Location: Fairfax, VA, USA
    • The Crowdsourced FIRE simulator
Re: cfiresim acting up
« Reply #28 on: February 18, 2016, 11:35:54 AM »

I do think the beginning of year withdrawal makes way more sense, and, from what I understand from most early retirees I've heard discuss their withdrawal strategies (primarily on E-R.org), is what they actually do.

Yup. This change was born from those discussions on E-R.org and the bogleheads forums (where excel spreadsheets are in heavy use).

Monkey Uncle

  • Handlebar Stache
  • *****
  • Posts: 1591
  • Location: West-by-god-Virginia
Re: cfiresim acting up
« Reply #29 on: February 21, 2016, 04:40:47 AM »
Quote
There are a few slight differences between the old and new, but that average ending portfolio difference seems large.

The biggest difference is how spending is taken out. In the old cFIREsim for any given year, here is how things are calculated:

- Jan 1 - Portfolio $X
- Dec 31st - Portfolio $X + Market Gains - Spending

The new version, takes spending out on January 1st (which most people seem to think is more intuative to how people would actually withdrawal) and thus that amount doesn't necessarily get the benefit of market gains. So, it's like:

- Jan 1st - Portfolio = $X - Spending
- Dec 31st - Portfolio = (Above amount) + Market gains.

All you're doing there is delaying retirement spending OMY.  Get it?

I think that the OLD method delayed spending a year, since it allowed the entire first year to accrue market gains before taking spending.


That should cause the old version to return a lower success rate than the new version.  But people are reporting the opposite.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #30 on: February 21, 2016, 04:42:06 AM »
No, if you leave the money invested longer, it would cause a higher success rate. The new version takes it out earlier, leading to the lower success rate.
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

Monkey Uncle

  • Handlebar Stache
  • *****
  • Posts: 1591
  • Location: West-by-god-Virginia
Re: cfiresim acting up
« Reply #31 on: February 21, 2016, 05:01:40 AM »
Sorry, I got turned around a bit.  Yes, delaying spending should cause the old version to return a higher success rate.  However, it is the new version that is returning the higher success rate (that's the part I screwed up in my last post).

Mmm_Donuts

  • Bristles
  • ***
  • Posts: 396
Re: cfiresim acting up
« Reply #32 on: February 21, 2016, 06:26:31 AM »
I had the same issue -- the new version gave me the higher success rate. I've been using the old version to be safe. It was a difference of about 20%, using the exact same numbers.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #33 on: February 21, 2016, 07:43:50 AM »

Sorry, I got turned around a bit.  Yes, delaying spending should cause the old version to return a higher success rate.  However, it is the new version that is returning the higher success rate (that's the part I screwed up in my last post).


Ah, I see. Good point.


I had the same issue -- the new version gave me the higher success rate. I've been using the old version to be safe. It was a difference of about 20%, using the exact same numbers.

Was this with variable spending, or a different spending plan?

Can you reproduce and post what numbers your using?

bo_knows can fix it, if it's a bug, if he has the data to reproduce it.
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

Retire-Canada

  • Walrus Stache
  • *******
  • Posts: 7739
Re: cfiresim acting up
« Reply #34 on: February 21, 2016, 08:13:58 AM »
Was this with variable spending, or a different spending plan?

Can you reproduce and post what numbers your using?

bo_knows can fix it, if it's a bug, if he has the data to reproduce it.

I had the same problem. I reported all the details using the "report problem" link on cFIREsim's opening dialogue box. This was back in Nov 2015.

Mmm_Donuts

  • Bristles
  • ***
  • Posts: 396
Re: cfiresim acting up
« Reply #35 on: February 21, 2016, 08:28:33 AM »
OK, just entered my current numbers and got a ~15% difference. It was 20% before, but I don't remember what the numbers were that I used.

Here's what I entered (anything not mentioned is left at default):

retirement year: 2016 end year: 2070

portfolio: $700,000
equities: 70%
bonds: 20%
cash: 10%

yearly spending: 72,000 variable
floor: 60,000
ceiling: 80,000

social security: 6000 start year 2031
social security 2: 6000 start year 2033

other income:

- 30,000 start year 2016 end year 2035 (recurring)
- 1,200,000 start year: 2035 (non-recurring)

The old version gives me a 85.71% success rate (failed 13 times out of 91 cycles)

The new version gives me a 100% success rate.

Pretty significant difference.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #36 on: February 21, 2016, 09:25:58 AM »
Thanks Mmm_Donuts.

I put those in and got the same results, 85.71% with old, 100% with new.

Interestingly, I changed the spending to the normal default, inflation adjusted, leaving everything else the same, and the old cFIREsim got a SR of 81.32%, while the new one, with that change, got 87.91%.

Once again, the new one is higher, when it should be lower, if withdrawals are now taken out at the beginning of the year.

So it's not just something related to variable spending, there's something more going on.

Thanks for the data, I'm sure b_k will chime in!
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

bo_knows

  • Pencil Stache
  • ****
  • Posts: 819
  • Age: 39
  • Location: Fairfax, VA, USA
    • The Crowdsourced FIRE simulator
Re: cfiresim acting up
« Reply #37 on: February 22, 2016, 11:42:17 AM »
Alright, so there are a lot of little differences between the old cFIREsim and the new one that result in a pretty big difference over a long period of time. This is a pretty deep "meta" discussion on which is the "right" way, but I'll give you my analysis. Please note that a LOT of the things that were done in the original (or now "old") cFIREsim were done simply because FireCalc did them, whether it was right or wrong. Things done in the current cFIREsim are done because after a lot of deliberation, they seemed to make sense.  Feel free to disagree and tell me why.

Before looking below, one thing to note: If you look at the defaults for each page, you'll see that the old cFIREsim had a start date of 2015 (yeah, I'm not updating it) and an end date of 2045 to denote a 30yr simulation. For the new cFIREsim it is 2016-2045, because of the order of taking money out and applying market gains.

Reference: Exact same runs, top is the "old" cfiresim, bottom is the current cFIREsim.

Differences:
  • It was pointed out that inflation should not effect spending for year 1, especially if simulating taking it out on January 1. So, the cumulative inflation in year 1 will always be 1 (which means no inflation applies). This has trickle down effect to mean that any portfolio adjustments (like the 30000 in the example) is not inflated,  spending isn't inflated, and coincidentally the final inflation adjusted portfolio isn't inflated. There needs to be some debate on this, but it basically eliminates a year of inflation, which could be a problem.
  • You'll notice in Column C, values are wildly different. In the "old" cFIREsim, you had all of your portfolio gaining/losing money ALL year, and THEN it took out the spending. Currently, we take out the money at the beginning of the year, and it then cannot gain or lose money during that time. This can have an effect down the line in simulations (good or bad).

This is probably the start of a longer conversation as to what "should" be done, and I am totally open to changing how this works, but we'd need a lot of consensus.

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #38 on: February 22, 2016, 12:06:49 PM »
Both those changes make sense to me, and seem to fit the "real world" much more.

I'm just shocked it makes that much of a difference.  Going from 85% success rate to 100% is a big difference.

Nice to know that the old cFIREsim was underestimating success rates then.

As an aside, when playing with them...I miss the GUI or graphic design or whatever you call it of the old one.

Also, minor feature request: is there any way to have a cookie remember that we've seen that particular popup, and not show it again, until it changes?  It may have to version number the popups or something.  Having to continually close it every visit is weird, it seems like that's something that should be remembered.

We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

bo_knows

  • Pencil Stache
  • ****
  • Posts: 819
  • Age: 39
  • Location: Fairfax, VA, USA
    • The Crowdsourced FIRE simulator
Re: cfiresim acting up
« Reply #39 on: February 22, 2016, 12:44:16 PM »
Also, minor feature request: is there any way to have a cookie remember that we've seen that particular popup, and not show it again, until it changes?  It may have to version number the popups or something.  Having to continually close it every visit is weird, it seems like that's something that should be remembered.

I'll consider that.  In the beginning, I really wanted it to show up every time you came to the site so that you could reference the old site... but I guess that doesn't make as much sense anymore.

I really need to overhaul the menus and links section too, as it's pretty bloated. I'm no UI designer, that's for sure :)

Also, in the new cFIREsim run with that above data, there are SEVERAL simulation runs that get down to like $150k portfolio. Seems reasonable that a few small changes way 50 years up the chain can make it swing just enough to make it a failure. *shrug*
« Last Edit: February 22, 2016, 12:47:30 PM by bo_knows »

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #40 on: February 22, 2016, 12:47:32 PM »
Oh yeah, forgot that's the main way to access the old site, I have it bookmarked.
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.

Mmm_Donuts

  • Bristles
  • ***
  • Posts: 396
Re: cfiresim acting up
« Reply #41 on: February 23, 2016, 09:11:12 AM »
Thanks bo_knows, for the explanation. Now it makes sense! This kind of huge discrepancy in the reports made me unsure of the calculations and so I thought it safer to believe the more conservative results. But now that I know the reason for the difference, it helps me trust the new version.  Having zero inflation in year one, and taking the spending amount out at the beginning of the year seem like good improvements to me. I'm a bit surprised it made such a big difference!

opnfld

  • Stubble
  • **
  • Posts: 133
  • Age: 45
Re: cfiresim acting up
« Reply #42 on: February 24, 2016, 03:10:23 PM »
cFireSim has a hard limit to 10 simulations it can run for logged-out users. After that, you'll always get the results from the 10th one, until you force reload the page.
thanks for pointing this out.  Is it in the documentation?

spokey doke

  • Pencil Stache
  • ****
  • Posts: 514
  • Escaped from the ivory tower basement
Re: cfiresim acting up
« Reply #43 on: February 25, 2016, 12:51:54 PM »
While Bo is here...can you say something about the relative merits of cfiresim vs. ontrajectory, which you link to from cfiresim???

arebelspy

  • Administrator
  • Senior Mustachian
  • *****
  • Posts: 28254
  • Age: -999
  • Location: Seattle, WA
Re: cfiresim acting up
« Reply #44 on: February 25, 2016, 01:18:16 PM »
While Bo is here...can you say something about the relative merits of cfiresim vs. ontrajectory, which you link to from cfiresim???

He talked about it a little in this thread:
http://forum.mrmoneymustache.com/reader-recommendations/ontrajectory-from-cfiresim/

Read that, and if your question still isn't answered, repost it with more specifics in that (more relevant) thread.  :)
We are two former teachers who accumulated a bunch of real estate, retired at 29, spent some time traveling the world full time and are now settled with three kids.
If you want to know more about us, or how we did that, or see lots of pictures, this Business Insider profile tells our story pretty well.
We (rarely) blog at AdventuringAlong.com. Check out our Now page to see what we're up to currently.