To make a judgement, we'd have to know more about your plan.
What's of paramount importance is that you not use the $38k you paid for your lot as an anchoring point for the second lot. That is, $20k isn't necessarily cheap just because it's lower than what you paid. You need to do a thorough comparison to other lots not only in your town, but neighboring towns. Be sure to factor in the HOA fee, realizing that a $300/mo annuity is worth on the order of $70k. So you'd be much better off paying $60k for a lot with, say, a $300/yr HOA fee. In fact, with a $300/mo fee, it's entirely possible that someone should PAY YOU to take the land off their hands.