I'll chime in with a different viewpoint....my wife and I have 'that house' in our circle of friends. No, not a ridiculously sized McMansion...what I mean is, it is definitely larger than we 'need' and is appreciably nicer in a better location than the houses almost all of our friends have. I guess my comments on it would be:
Every family that chooses to have a nicer more expensive house isn't automatically financially worse off, it's all about prioritization. We spent more on our house than many people we know, but we also are well below the 'mean' in other areas and have old cars, aren't outfitted in expensive clothes, I bike to work, and we cook most of our meals at home, etc etc. We are home-bodies who like to entertain so that's where we chose to spend our money, I don't see anything wrong with it and we still save ~50%, we just gave up enough stuff in other areas of life to get the house we wanted without compromising other goals.
So OP if you really want a house like that as your chief desire, try and carve out enough other stuff, keep saving, and get it! I'm not ashamed to say it...having a nice house does make me happy (personal decision), and I think that's a good chunk of what saving money is for...to enable things that make me happy. YMMV.