Many people purchase their first home as soon as they can afford anything, then sell/buy/switch homes every several years by moving cities or upgrading. These moves incur huge real estate fees, moving costs, and, worst of all, reset the mortgage clock. The first 10-15 years of mortgage payments are mostly interest. Start a new mortgage and you are starting over at the bottom.
If you want to optimize this part of your life, wait a little longer until you have the perfect house in the perfect location in the perfect city. But that and stay forever. Warren Buffett's modest house in Omaha is a good example. He could afford something grander in say, St. Tropez or Central Park West, but he stays mostly in Omaha. Many people get a little money saved or maybe an inheritance and then the urge to "upgrade" or make more room for kids. Never fall into this trap. If your first house has three bedrooms you are set for up to five or six kids. Three kids can easily share a room even in high school. And if you have that many kids, you definitely can't afford a larger home as you need the cash for food and clothes.
I see a lot of people fall into the trap of upgrading homes after the kids move out as well; some are the same idiots who bought a five bedroom house and now want to downgrade to three or two bedrooms. Another huge waste of real estate fees, moving costs and potentially restarting at the bottom of the mortgage ladder. These people are working an extra 5-10 years to afford all this real estate swapping.
By avoiding these real estate pitfalls (I've made plenty myself and watched others who did the same), one can minimize living costs and allocate all the cash for other more valuable things.