I flip flop on this -
Our house is the perfect size (750 sf) for my wife and I. We are in a nice central location, low maintenance, easy to clean, low utilities, etc.
We have just enough room to have our friends and family stay for extended periods without being too crazy, but not much privacy.
2 years later we have a kid in the way with the prospects of having one of our parents at the house off an on for a year perhaps. That second bedroom is looking really great all the sudden. DW is working from home also.
We could have bought something for twice the price without really stretching ourselves much, but I wanted as little overhead as possible. Seems short sighted now.
Other days I am grateful we have a sensible space that is just big enough for our growing family. We can make it work without needing a mortgage which gives us flexibility in other ways. We didn’t need new/more furniture to fill rooms we didn’t have before in our 350 sf apartment.
On the next house I plan to splurge a bit - I want 5+ acres and a custom home. I’ll probably shoot for 1500-1800 sf + a large detached garage. I’ll spend the money I didn’t spend this time anyways. So it’s not like I’ll never own a house that expensive, I just postponed it.
Other thoughts - transaction costs are high, 5-10% depending where you are, and moving sucks. If you are where you want to stay for 10+ years I say buy the forever home. If you are unsure what happens in 5+ years then get something that is easy to sell, logical to rent, and cheap to own.
The whole buy small, upgrade, upgrade, upgrade, downsize model of homeownership assumes you live in a rapidly appreciating market.
Final thought - there is something great about using as little resources as possible. Sometimes small is beautiful. We waste a lot - but that is not guarantee that your future spouse will feel the same, although you can select for that.
Edit: location, location, location - look at school districts, hospital grades, crime rates, proximity to groceries, churches, etc. The perfect house in the wrong location is not great either. Consider your needs as you get married, have kids, parents get old, etc. Maybe your life doesn’t require all of those considerations, but it is nice to have options later.
Think about things that will annoy you - heavy traffic, on a fire station route, air plane noise, train noise, etc. maybe some of those things don’t bother you, but it could mean a spouse hating your perfect house if they do not share the same tolerances.