This is not a zoning restriction, but some HUD guaranteed reverse mortgages of multifamily (topped out at fourplex) residences require continuous owner occupancy of one of the units. That would be one half of a symmetrical duplex for the finger-counters out there. Any period of non occupancy (by the mortgagor) longer than twelve months is forbidden.
The penalty for non-compliance is mortgage default and foreclosure, sale of the property at auction.
Invasion of privacy or not, there it is. And who wants a battle with the mortgage company and HUD?