I think the numbers are close. Let's say property is increasing in value over the long run by about 2%/year; so that's another $10k a year gain for you. Not knowing the house, maybe you ought to budget $4k a year for maintenance? So you're effectively getting your $13,300 "profit" plus $10k appreciation minus $4k maintenance = $19,300 a year on a $500k investment, so just under 4%. But it requires some work from you; collecting rent, accounting, getting tennants, contractors, etc.. You could figure the value of your time and back that out of the equation. And you are somewhat at risk for a loss if a tenant trashes the place or you go vacant for a little while, the water heater leaks etc.
Maybe other folks would make different assumptions (appreciation, add in the value of the house's depreciation tax writeoff, etc), and I obviously don't know your market, but I'd evaluate it in that framework - what rate of return is the house getting, and will stocks do better than that in the long run. Honestly, the risk of depreciation in the stock market is probably similar to the risk of depreciation in your house; i.e. if the stock market goes down 20%, your house probably will too.
If you're really going to live there or somewhere exactly the same (and you're OK that your retirement housing is making you 20k-25k poorer per year) probably you should keep it since the transaction costs on that much real estate would be in the $25k-40k range (I think?) for the buy+sell.
But if there's a good chance you'll see the light and move someplace cheaper, I personally would sell and invest. Long run, stocks are set it and forget it and should do a little better than the 4% anyway. When rates of return are close, I'm a fan of applying Occham's razor to the investment decision; the simplest investment is the best one. In this case, where the returns of real estate aren't obviously better than stocks, the simplest investment (stocks) is preferable.