We're much like you. We earn a little less, have a little more in net worth, and I suspect are a little older. Our oldest is just finishing college, and our youngest is about to finish freshman year. Our oldest is finishing on time, debt-free, and has a good job waiting; our youngest is on track for the same.
Our kids both earned some nice scholarships, and we've had no problem cash-flowing the rest. Really, it's been easier than we expected. Even without the scholarships, we would've been able to do it -- though it would've required more effort on our part.
We had a backup plan: We were prepared to stop making contributions to our retirement plans. Why? Because we're close to retirement, and our best savings years are behind us. We saved heavily when we were younger, and we are happy with where we are. If we hadn't done that; that is, if we were playing "catch up" at this point, this wouldn't be an option.
Let me be clear: We were NOT thinking about what you're thinking. We were not talking about taking already-socked-away money out of our accounts. That's problematic. You pay fines, it messes with your taxes, and it's just not a great idea.
But, no, if you need 20K over the next four years, I don't think taking from your 401K is your best bet. You need roughly $400/month between now and then. You can do that. Actually, you have four years to come up with 5K ... five years to come up with another 5K ... six years for the next 5K ... and seven years for that last 5K.
One thing to consider: Is he your only child? If you have other kids, plan to do the same thing for them. I've known a couple families who've moved heaven and earth to make sure the oldest child could attend "the school" and have "the college experience ... but then by the time the younger kids come along, they've realized the error of their ways, seen the debt they were incurring ... and they've told the youngest he's going to community college and should be grateful. Be sure each child has equal opportunity. They may not make the same choices and/or may not step up to the plate equally, but be sure they both have the same chances in terms of your help.