Hi, and welcome from another Aussie.
The bond market in Australia is far less developed than the bond market in the US. It is difficult to access many bonds directly, and there isn't much of a secondary market (i.e. its difficult to sell your bonds).
If you want to buy government bonds directly, one way is direct from the reserve bank. See
http://www.rba.gov.au/fin-services/bond-facility/ for their information page.
An alternative that might offer a bit more liquidity for you is to consider some of the bond exchange traded funds that trade on the Australian stock market. These are easy to buy and sell through any online broker. Have a look at VGB for a government bond fund provided by Vangaurd - one of the worlds largest providers of managed funds and exchange traded funds. More information on that is here
https://www.vanguardinvestments.com.au/retail/ret/investments/etfdetailVAGBIE.jspHowever, be careful - bonds are different to high interest saving accounts, in that with a bond the capital value of the bond can rise and fall (and does so with interest rates). Typically, if the interest rate falls, the value of a bond will rise. If interest rates increase, typically bond values will fall (with the potential that you can lose money).
Bonds have been going up strongly in value over the last couple of years as interest rates have crashed. My personal view is that there is a significant degree of risk in buying bonds right now (google "bond bubble" and you will find plenty of conspiracy theories), and my personal evaluation is that at the moment shares are potentially lower risk (and that's where I have my money). If the risk of loss of capital worries you, I suggest that you should do a lot more research on bonds and how they work before buying any!
Hope that helps