It was zero annoyance -an absolute pleasure, in fact- to manage the rentals while excellent tenants were in them. I loved advertising, doing the forms, knowing I was providing lovely housing for people, responding to the rare need, being legal and beyond fair.
It was an extremely high annoyance to manage rentals where applicants or tenants were awful. I hated facing damage to the property I'd cared for, having to call police, having to remind people to pick up poo or to do it myself, etc.
I had extremes: excellent tenants, awful tenants. Same screening process for each: live interviews, reference checks, criminal record checks. If they had a pet, I required them to bring it to the interview too. All around the same ages, many students, all studying or employed.
Excellent tenants?
Quiet, took care of their spaces, paid rent on time no problem, gave legal (or at least reasonable) notice.
Awful tenants?
One was awesome, until she got drunk, which turned out to be at least several times per week. Then she would abandon her toddler, bring other people into the house, leave them there and go out. Shrieking music, someone arrested, someone left behind screaming for some hours until she passed out. Etc. Another was awesome until she got into a car accident and was in too much pain to care for her very large dog, which became a problem of poo everywhere. A third one found someone living on the street, so moved him into the shed and charged him rent while he damaged the place, then she moved out without paying rent to me and leaving him behind with a key to the house, which meant I couldn't necessarily move him out without process.
Challenges?
In BC, the (legal) eviction process is pokey and slow, and the landlord and property are at risk in the interim.
Some people give out friends' cell phone numbers as work and character references, and the friend pretends to be the employer.
A landlord having trouble with your applicant may encourage you to take them so that they can get rid of them.
Tenancy laws here also dictate many other aspects: can't charge more than half a month's rent for security, so managers may feel a need to scramble when someone flees with no notice, etc.
On my side?
Besides the good tenants that started out as strangers to me, I had friends to rent to for long stretches.
A low vacancy rate, plus relatively low rents, plus willingness to accept children and (some) pets meant I had larger pools of applicants.
I had neighbours that paid attention and called me when things were weird, so I could act on things quickly.
I worked within tenancy laws to successfully negotiate for some to leave on mutual terms, no permanent damage.
The managers in BC that I knew to do well were willing to break laws (circumvent tenancy laws, threaten tenants). I wasn't willing to do those, and was going to be further away, so I got out of the business.