For what it's worth, here's my family's (n=1) experience on house sizes.
The first house that my husband & I lived in as a married couple was adorable. It was a 1930's home in an established neighborhood with loads of charm. It was 1500 sq ft, which seemed perfect for the two of us. Once we had our daughter, the house started to feel a bit small.... less due to our daughter and more due to the fact that we then had a daughter and two large dogs, one of whom could not be trusted around the kiddo. The house didn't have a good way to allow effective segregation of the dogs, it had the steep/narrow stairs and wasted hallway space typical of older homes, and we decided that we needed to go bigger.
We moved from that house in a 5bd/3ba 2500 square foot house. Yes, really. At first, it seemed great.... we had an office, a guest room, a bedroom for our kiddo and her future sibling. We also had extra space in the form of a sunroom, a weird sitting area off the living room, and a weird hallway. And the biggest guest bathroom you've ever seen. We loved it... for about a year. And then it started to feel like every spare moment was spent cleaning that big, stupid house. The house needed some repairs/updates, but the size of the house made it crazy. For example, we wanted to repaint the whole thing.... but the living room ceiling was about 30 ft high, so we would have needed to hire it out because we weren't prepared to take on that sort of task. We had rooms that went unused and basically just collected junk.... which meant that periodically cleaning those rooms out was a huge PITA.
We downsized to a 3bd/2ba 2000 sq ft house. This house is just right. We still only have one kiddo (trying for a year with no success for #2, although admittedly not being very aggressive in our attempts, and starting to worry that maybe I'm too old). The third room is a guest room / office. If we have a 2nd kid, the 3rd room will go to the 2nd kid, the desk will move into the master bedroom (specifically looked for a master big enough to make that work) and the kiddos will share a room when we have company in order to give up a bed for guests. (Or I'll someday successfully convince my husband to let us enclose the never-used formal dining room to create a guest room / office, which is unlikely.) Our bills are lower, we can repaint the house ourselves when we're ready to tackle that, and I can vacuum the entire house in about 30 minutes. Love it. I honestly think we could have gone even a bit smaller, but this house was the best fit for our criteria (close to his parents' house for free babysitting, right school district, in a neighborhood, right price).
Again, we're just one family.... but we definitely learned our lesson that less is more when it comes to houses. (We also downsized from 5 acres to 0.5 acres.... and with what we save on lawnmower gas/maintenance, we pay a guy to mow our lawn for us. Best decision ever!!!)