The Denver/Boulder area might be a red flag. I spent some time house shopping in the Boulder county area, and had some interesting conversations with locals and builders. Apparently, Boulder county has some pretty over the top regulatory burdens, that far exceed what is expected in most parts of the country, and costs have skyrocketed. Don't know exactly how bad it is, or if it extends to other counties, but I heard some pretty wild prices for even basic "builder grade" construction, based on what the ever exploding market will bear, and the silly amount of over-regulation. So, look before you leap. Good luck.
As a personal note, I built custom homes for a living. My latest personal dwelling is for sale, and we are relocating. I'll find a place to flip when we find our next location, and wouldn't even give a though to building one from scratch. Finding a new set of subs, and dealing with another bureaucratic maze is not something I'm willing to put my self, or my wife, through again. There are many happy owner builders out there. There are also just as many uncompleted (and often occupied) houses out there, an broken marriages caused by the stress of "saving a whole lot of money by building our own home". If you do it, sub out whatever the budget will allow, and if you are not a tough skinned, cynical prick, you better be able to fake it, since it isn't a task for the weak.