well, the % growth is the same regardless of the stash size.
Not for me it isn't. The vast majority of my passive income comes from my retirement-sheltered tax accounts, which are heavily invested in stocks.
In 2013, my passive income was ~115% of my (gross) earned income.
In 2012, my passive income was ~50% of my (gross) earned income.
Believe me, the changes in proportion do not reflect either significant changes in my earned income or (particularly) significant changes in the underlying portfolio value.
And while it's been -- knock on wood -- some years since my passive annual "income" was -100% of my annual earned income, I'm pretty sure that's happened too.
To answer the OP's question, I'm about 15 years into saving (I'm not particularly interested in RE, so "how fast" isn't as big a concern for me as it is for many here). I'll note, though that my first 2 years' worth of retirement savings took 13 years to double (This was easy to track, as I left a small 401K at my first employer whom I left after my first two years' of work. In 1999, the account was worth about $10K. In 2012, it hit $20K). In contrast, the (different) $60K 401K I rolled into a Roth in 2010 is now worth $102K -- so, it hasn't quite doubled in 4 years, but it's not nuts to think it might, by 5. Or not, we'll see. Contrary to what all the "annualized" descriptions will tell you, when you start (something you cannot readily control) and how long your time horizon is (something you typically more readily can control or at least, adjust), matter.