The reference to slow progress on the build suggests to me that the initial 3 year period is close to running out, so that advice to pay aggressively for the three years is probably not applicable.
The first thing to do is a full evaluation on keeping the land at all. The failure to make progress, coupled with the question about making money off the land as it currently is means that the build is not an immediate priority. Also, the fact that the loan just to buy the land is creating a financial issue makes me wonder about where the money for the build would be coming from. In those circumstances the need is to make a serious plan for getting the build done, including financing and time allocation. If there isn't a way to make it work in a known time span the better option could be to sell the land and try again when finances and time issues permit.
If OP is serious about keeping the land even without being able to get on with the build, they need to crunch the numbers as to how much it is going to cost them to keep it over say the next 5 years. What's the new rate on the existing loan? What are the costs of getting a new commercial loan, both the interest rate and the closing costs? The problem with the 401(k) loan is partly the need to pay it off immediately if you leave your jobs and partly the fact that you are taking your money out of investments for it, which means that the cost is the compounding loss of market returns on that money over the period of the loan. It's a hidden cost compared to a commercial loan which is why it looks to be potentially a better deal: it really isn't.