We've got 2 adults, 3 young kids (all girls - 5, 3, and 3) in 1850 square feet right now and we're continuously on the lookout for a smaller place. We've got all 3 kids in one room and they love it. Not to say there's never fighting, but they really enjoy sharing the space.
We're working on updates throughout and at times we've been down to essentially a 2 bed 1 bath house with 1100 sq feet and were able to make it work no problem for weeks at a time. In the end, there's a lot to be said for trying things out before you become convinced that you "couldn't possibly" handle living a different way.
It's hard to find smaller homes in our community but tough to move as well - we love our church, the local pizza place, the school district, and have great friends nearby, but most of the houses are our size or bigger (in some cases significantly).
We just keep crossing our fingers that something will pop up on zillow that meets the criteria. We're not afraid of fixing something up but the right opportunity just hasn't come up yet.