You have no control over the operations of your company. Your CEO earns millions of dollars while he sweeps some minor fraud under the rug. You don't get to use that corporate jet your management team just purchased. Are you aware your COO has a cocaine habit and repeatedly abuses your employees?
But I think I have a lot of control. I have the SEC, SOX, governance, and so on. I know how to read a balance sheet and a QR. I know exactly what my officers are getting paid and I indirectly control their pay and employment status through my BODs. I think the mistake people make is lumping my company in with Wall Street. I love owning my company but I hate Wall Street. To generalize all of them together is to miss out on a great investment opportunity. I understand the risks you stated about my CEO and COO, but they are less risky than your tenants and lenders because they are forced to be transparent.
I think if you have deep knowledge of any income producing "thing" you can find the sweet spot. If you are sure that a company is well-run and in a good profitable market you can control a lot of the risk. That takes a significant amount of due diligence and fraud is always a possibility, as are black swan events.
I do agree that if you have that degree of knowledge you can likely adequately mitigate risk. I think this is a strategy that Warren Buffet follows?
However, unless you are buying on margin/credit and using leverage to the same degree and the same lending rate as you would when you buy a house, you will most likely not make the same return on your initial investment. I would not be comfortable with leveraging stocks myself.
I definitely don't have the same deep knowledge as you do about stocks, but I do understand real estate in my local market. I know every street and what the market value of a home is, whether the house can be suited, and any neighbourhood pros and cons including non-obvious ones like seasonal flooding or subsurface earthquake vulnerability.
I also know my credit, mortgage rates, expected rents, taxation issues, how to evaluate replacement costs and timeframes for housing components, and vacancy rates. I can calculate expected ROI in my head now and be fairly accurate.
Mostly the numbers do not work where I live, but once in a while there is something that does. In the past five years I've found three and purchased two of them. I bought a third prior at a time when I had less knowledge and it does not perform as well.
I don't treat my primary residence any differently. It needs to be cash flow positive too. Your credit is limited and failing to make this significant source of leverage work for you seems like wasting an opportunity to me - at least prior to retiring anyway.
As far as lenders being risky, I don't see it. As far as tenants being risky, that is what screening and written agreements are for. You cannot control for everything, but you can control for most things.