It is possible to build for much less than you can buy for; we did it. However, it's not simple. How much time do you have? If you both work full-time and don't have anyone else willing to volunteer labor, I'm not sure it's possible. It's a tremendous, tremendous job and it will take much longer than you think it will.
We built ours with both of us teaching, so we had summers off and spring breaks to work. Also, my father had retired, and he spent many months working alone during the day until my husband got home from school to help. He generally stayed here four days a week and went to his home for long "weekends" (usually during the week so that my husband and he could do heavy work together on Saturday and Sunday). We didn't have weekends or any other breaks during the year and a half of building. The exception was when the weather was too bad to do anything (think active thunderstorm) and as we made more progress and got the place dried in, there were no exceptions.
My father built the house I grew up in while he was working full-time. But, he worked nights, 12 hour shifts, and on his "weekends" he left work at dawn, went to the build site, worked all day without sleep, then worked the next several days while he camped at the build site, then went back to work for the night shift without sleep or with just a few hours' sleep. I was a small child, and my mother and I saw him only during his work weeks, when he came home to the house we were renting to sleep during the day.
I don't mean to be nearly as discouraging as I'm afraid this sounds. It is an incredible accomplishment, and it feels worth doing during the sacrifice and afterward. There's no way I'll ever be sorry, and 40 years later not only is my father not sorry about his place, but he's not sorry about ours or the one that he and my brother built for my brother's family a decade ago. But it's not possible to overestimate the amount of work. You need to factor that in as you're making a decision. Also, you have to be absolutely sure you want to stay forever. You will not be willing to move out of a house you built with your own hands.
For us, it's meant an incredible jumpstart toward financial independence, and it almost certainly made the difference between financial independence and not having enough for my parents, but it's a lot harder to accomplish than simply saving and investing money.