My advise, you need to take out a loan as soon as you are eligible to. With the rates where they are, you don't want to miss them. How much is entirely up to you, but I'd at least take out what you put in to build the place.
If you are looking to expand, having the money ready and available makes it that much easier to act when/if a deal pops up. You don't want an off market deal to present itself and then have to deal with getting financing from your place to finance the new place. Being able to say i have XX to put down (or pay it in cash, to refi later), makes your offer strong.