I follow a permanent portfolio type strategy and currently hold the GLD index. You can also use the IAU. I plan on moving to physical gold in the near future. I will use a safe deposit box, as you mentioned they are pretty affordable if you hold an account with the bank (although it depends on size). I don't see much downside to using a bank, they are pretty unlikely to be robbed for their safe deposit boxes (think of the time involved with opening and clearing all of them). In the unlikely event that there is some huge financial crash and banks put a freeze on accounts, I would be willing to bet you can still access your box. I think it is pointless to try and get involved in some complicated storing mechanism unless you are holding more than would fit in the largest safe deposit boxes.
www.goldmart.com is where most people that I know do their buying. If you buy bars, there is very little premium over spot price. If you buy bullion coins, there will be a premium anywhere from $30-70 depending on the coin. They also have free shipping for large orders ($7500 or more).
Coins:
Pros: Will sell for over spot, can be used as actual currency in a disaster (ie, a gold eagle is US currency with face value of $50), typically easier to cash in/sell, many people deal/collect them. The most liquid coin is the American Gold Eagle, followed by either the American Buffalo or Canadian Maple Leaf
Cons: Higher premium, more small pieces to store
Bars:
Pros: Very little premium over spot, some have some pretty intricate patterns and look very sharp
Cons: Harder to unload/sell, zero value beyond the metal itself
