It's definitely worth it to me - I keep the cash, no free money for my lender to hold. Like others have said, insurance is easy to pay via CC and two of our properties allow CC payment with no fee.
I did just screw up for the first time this year. Forgot about following up on one payment until the day after it was due. The county assessed a 1% fee. That sucks, but it was still (barely) covered by the 3% CC cashback, less this county's 2.5% credit card fee. I am not worried that will happen again.
Plus, I've had a tax payment fiasco when the lender was doing it via escrow. The county said they didn't get paid and the lender said they did. There was very little I could do, and the lender wouldn't or couldn't provide some kind of evidence or details of payment - as I repeatedly told them they'd insist I provide if I claimed to have made a payment they didn't see. The lender doesn't actually send a simple payment for each property, they send one big payment for all their properties in the county with some kind of ledger noting how much is for each property. It eventually got sorted out with no hard costs, but it was a huge time sink for me.
So that's two goofs over multiple properties and over 20 years, and the lender's was far worse to deal with.
I'm a cheapskate, so I never paid fees to avoid escrow when getting a mortgage, but a few allowed it with no extra cost, and almost all allowed removing it soon enough. So I've had much more time without escrow than with.