So first, do you track expenses? There are a lot of categories of things not listed here -- clothing, kid events, gifts, car maintenance/new tires/registration, home repair/maintenance, restaurants/entertainment, life/disability insurance, etc. If you don't track, you need to start, so you can get a handle on all those little things -- otherwise you end up with death by 1,000 cuts.
Second, there are a number of things that you should be saving for monthly to ensure it's accounted for in your budget, so you have the cash saved for when the expense occurs -- I think of it as a slush fund. The big ones from your post are a replacement vehicle fund and a vacation fund. It's great a car will be paid off in 2024, but that car won't last forever, so you want to be saving for a new one so next time you don't need to take out a loan (or take a smaller one -- really, by your age, I'd have wanted to see a car fund sufficient to pay for the next car in cash, but I can imagine the cost of 3 kids would get in the way of that!). This category also includes less-predictable things like a new roof for the house.
Third, do you have a budget/plan for the new home build? If that is part of the short-term plan, you need to understand how much house you can get and how much it will cost. Frankly, I'm concerned about whether your plan to build is feasible, at least alongside your other goals. Your mortgage is very, very cheap; even if you took out a new mortgage for the same amount now, it would cost a lot more. Assuming you get the land paid off before you build, you either need to fit your house costs into that combined $1100/mo. that you're currently paying, or plan for much higher housing expenses. FWIW, my loan calculator says that at today's rates, a $175K house with 20% down wil cost you about $1150/mo., making some assumptions about taxes/insurance. So if you want to maintain the same costs, that means you need to save about $40K (with closing costs/etc.), get the land paid off, and then build no more than a $175K house.
Which brings me to my next point: building costs are wicked high. They have come down some since the Covid-induced supply chain hell, but at least in my part of the world, you can expect probably $200/square foot for a relatively low-end build. And unfortunately, many of the things that make a house cheaper to run long-term/better to live in (like foam insulation, highly-efficient boilers/solar systems, quality windows, etc.) have bigger up-front costs. So I am concerned that the kind of home you want to build will either cost a lot more per month or require a lot bigger downpayment (or some combination of the two). Which means, if that is your priority for sometime within the next 5 years or so, you need to start assessing the likely cost, figuring out what you can afford, and cutting expenses/saving the delta so you can afford the house you want when the time comes. Alternatively, if that analysis shows the dream home just isn't realistic given your other priorities, you can sell the land and save yourself basically 10% of your take-home pay every month.
Ultimately, the key here is going to be for you to decide how to balance your priorities. You have a lot of really good priorities here -- taking care of your family, enjoying family travel, financial security, a nice house, etc. But a bunch of those are potentially big-money -- and you have only so many dollars, and you can spend each dollar only once. If you want to end up in the happiest place, you need to find an appropriate balance between CurrentYou and FutureYou, and you need to then prioritize which of those current/future goals are highest priorities. So, for example, you may decide that being able to travel with your kids while they're young is a higher priority than the dream house. In that case, you defer the house fund and devote the savings to vacations and other priorities. Or vice-versa -- if having the new house for the kids to grow up in is your absolute highest priority, then you cut vacations, entertainment, clothing, eating out, and all that other non-necessary stuff so that you can get that house within the next couple of years. You're the only one who can decide what those priorities are.
