The math is not too tricky. The money you use to pay down your loan means you incur less interest. That means you are getting a 4.63% return on your money. However, this benefit is post-tax; you do not pay any additional tax on the money you save. If you were to put your money in investments instead of your loan, you would need to pay tax on the returns from that investment.
Assuming the yield from your investment is paid out each year (term deposits etc), the return % at which you would be indifferent between paying down debt and buying investments is 4.63% / (1 - your marginal tax rate %). So if you're earning $50k a year and thus in the 34% bracket (including medicare levy), you would need to earn 4.63% / 0.66 = 7.02% on your investment or better in order for investing to be a better option than paying down debt.
It gets a bit trickier if your investment is held over time in the form of capital gains (real estate investment trust, shares that don't pay dividends, etc). For these I'll assume you hold the investment for at least a year, so benefit from the CGT concession, halving the marginal tax rate that applies. Therefore, for these investments you would need to get a return of at least 4.63% / 0.83 = 5.58% or better in order for your investment to be a better choice.
This math assumes two extremes: 100% capital gain or 0% capital gain, and does not address the riskiness of investing. In short, equities and bonds will get you better returns on average than you would get from paying down your mortgage; that said, you're also taking on the risk your investment may lose value, and added complexity in managing your investments and tax affairs.
My personal opinion is that outright ownership of a property yields such gigantic peace of mind that anyone who is not a knowledgeable and risk-seeking investor should pay off their mortgage first.
EDIT: An offset account is a brilliant idea.
I have a Vanguard account but I am saving towards a deposit and have no commercial debt, just HECS. Once I can afford to buy I will focus on paying off my mortgage first.