I would hold an appropriate asset allocation for the timeframe you're considering. 4-5 years, in my mind, means you're holding a combination of stocks and fixed income, probably anywhere from 15-50% stocks in your down payment fund, depending upon your risk tolerance (which means, in this case, your flexibility for shifting your purchase timeframe and house price).
I wouldn't buy the house early unless you have a good plan for being a landlord while you travel. Holding an empty house for years would be a huge waste of money.