I suggest you start looking but not get carried away to make the move immediately. Take time to research and identify the neighborhoods and house/property features you really want and familiarize yourself with the market. Continue to save in the meantime! Then when you see a house that you know is the one, you'll be in a good place to make the move with confidence. Also think a lot about he extra work and expense that comes with the bigger place. You'll miss the savings you have now and grumble about the extra house/yard work on the weekends. Having said that, I think wanting a bigger place for a family of 5 is not unreasonable.
Here is my experience:
Ten years ago, I was making $100k and had a $400 mortgage payment on a 3BR 2 bath house, probably about 1200 sq. ft. We are a family of four. We were saving a ton of money! The neighborhood started to decline, the house had some issues (or I'd have kept it as a rental), and we wanted more space so we moved up to a 2700 sq. ft. 4BR 2.5 bath brick house and a mortgage of about $1400. Since the old mortgage was so cheap, we waited until after the move to fix up and sell the old house, making for a very leisurely moving experience. Somewhere during the move, we picked up a bad habit of eating out too often that has never stopped. Other than wasting too much on food, we continued to save and invest but I missed that cheaper living, though loved the new house. I should admit though that we didn't use all that new space. Spare bedroom never became the craft/hobby room like we planned and we never use the dining room. 2-car garage has kayaks, bikes, grill, etc. on one side so I still only hve a one car garage. My wife has some hoarder in her so a lot of the extra space in the house is filled with junk. I guess we tend to fill the space we have. I did pay off my mortgage two years ago when a layoff seemed imminent and while I greatly enjoy not having a mortgage, I freely admit that I'd hve been better off financially to refinance from 4.75% to 3% and keep that money invested as the S&P was 1200 at the time and just hit 1800 this week. While I still like our house and 2700 sq. ft. Is not huge, a smaller place that's closer to work and easier to maintain for the same price would have been a smarter choice.
With 3 kids, it sounds like you can justify the bigger place. Just don't rush into it. And use your mustachian skills to ensure you spend the money on location, efficiency, and low-maintenance, not just more space that will get filled with stuff.
Good luck!