I had been ignoring tax, which doesn't really change the result, but it makes it different.
If you're game.. read on...
I'm in Australia, so I don't get a tax deduction on my mortgage, and there's this thing called dividend imputation - which means, a share of the company tax that is paid, gets passed onto share holders, and it reduces the tax you would have to pay yourself. Effectively, if your personal tax rate is the same as the company tax rate, you effectively don't pay tax on most dividends. 'Unfortunately', I'm a bit above that, so I pay about 10% tax on my dividends, so 5 goes down to 4-4.5% (I'm not quite sure), so my example above doesn't really hold.
But it still does, because for many of my shares, I've borrowed money to pay for them, and effectively that interest is a tax deduction (those shares are about half at 4.5%, and half at about 7%, call it an average of 6%) so for them, I don't pay 6% interest, its more like 0.6*6 = 3.6%. So for those shares, even with the dividend tax, I would have worse cash flow if I sold them, and paid off the mortgage.