People live with much less space and fewer amenities throughout the world, so technically it could work.
If you haven't seen the book Material World, by Peter Menzel, it's full of photographs of families from countries around the world taken with all of their worldly possessions. It's fascinating.
For me, the problems start when you can't find a quiet place to escape. It's not so much the kids' crap (which you have some control over) as it is the noise and chaos.
My kids are ages 4 and 8 and we live in a 1250 square foot 1920s bungalow. It was such a comfortable house for 3 people + 1 dog, but once we had our second child things got crowded quickly. We only have 1 bathroom, which is o.k. as long as everybody isn't leaving the house at the same time.
My son and daughter share a room, so if my daughter wants to have a friend spend the night, my son has to sleep in our room. If he gets up at 6:00 in the morning, he usually wakes her up.
If the kids are watching tv or listening to music, the only place we have to escape is our bedroom or outside.
The kids also share 1 small closet and 1 dresser, which was fine when my son was a baby/toddler, but is not ideal anymore.
These are all things that we could deal with for 3 or 4 more years, but we want to stay here forever, so we're doing a small addition.
You'll know when the time is right to move on.