For certain jobs, like teaching, the masters is a huge help. Most professional government jobs you just straight up get paid more because of the credential. Needs to be weighed against the cost and time though.
That said, that credential is specific to teaching, so it would only be worth something if you did teaching.
The advice I've heard, is that the only reason to become a teacher is because you think you'll really enjoy teaching. It happens to be convenient with kids because the summer childcare isn't a thing because you're off when they're off. And the teaching salary does actually get decent if you are willing to go ahead and teach in the summer and can ditch the kids with family for free (depending on how they do that in your area). You'll also have access to government retirement savings things, which tend to be better than what you can get outside of government.
It is a difficult choice you face. If it were me, it would have to wait until the youngest was off to school. I don't know if there are deadlines you run into for some of the things you were talking about that might help pay for school, so maybe that offsets the childcare.
I would take a look at what the actual requirements are to teach what you want to teach, it is possible you could pass the certifications and so forth to become a teacher now. I know in Texas they used to have something called an "emergency teacher certification" or whatever, one of my sisters taught for a year using that, while looking for a job in her actual field, no extra coursework beyond a b.s. was required.
My mom did what you were describing regarding a Masters, going to night school while we were kids, to enhance her teaching career, and I doubt it paid off before I left the house. As in, the cost of the education probably took more than 15 years to justify itself. It isn't just an economic boon though, there's certain quality/stability of life aspects to bettering yourself. And it certainly left an impression on us kids as to how important education was. Also made us appreciate mom's cooking more (seriously dad? chili? again?)
The additional income from the new job has to offset:
1. Daycare
2. Commute costs associated with job
3. Any spending control you have now that you wouldn't maintain because of stress/time (food, clothing, housework, coupons or w/e)
4. Any personal dissatisfaction with working (some people actually hate being home with the kids more so this is a personal preference thing)
I value a half-assed homemaker at around 50k/year. Someone who is hardcore into stretching every dollar, taking care of every possible task including taxes/investments/bills as well as all upkeep at closer to 100k/yr in terms of the value they provide to the whole family organism. And that ticks up with the more kids there are/more income brought in by other spouse.
Just something to think about.