I think that Opportunity Zones are kind of like LIHTC (Low Income Housing Tax Credits) where the biggest thing that can go wrong is that you violate the program's requirements...that is to say the biggest cost/downside is going to be due diligence over the 10 year period...on top of the start up costs.
I think the biggest thing that can go wrong is that you lose money. I've looked at the maps for my area, and as a current landlord there is no way I would want to own a rental in any of those places. Not unless I'm trying to own a condemned meth lab.
At least around here, those appear to be areas with rampant crime, no public transportation, the worst possible schools, and horrendous exposure to natural disasters. Much of it is what I consider "disposable" land, the kind of place you know will get wiped out by the next volcanic eruption or major flooding event, only suitable for trailer parks and expendable low-grade industry. It's not where you would choose to build anything you expect to last a hundred years. Maybe it's better where you live?
If I were to buy a rental property inside the closest zone to my house, I would expect to deal with evictions regularly. I would probably have to make it section-8 housing in order to find people who could afford it, and that comes with all kinds of other headaches. I would just assume there was illegal activity taking place on my property, and I would constantly worry about vandalism and arson. Frankly, I'd rather pay capital gains taxes. I already have enough money that I'm willing to spend some of it to reduce the amount of stress in my life.