I belong to two different departments at work, and each have a gift swap. That makes it very easy to opt out of both without anyone noticing. For other types of events where you are obliged to participate in this type of things (lotteries at the kids' school, etc), we have a "gift box" at home where everything that is up for re-gifting goes, and where we pull prizes from. Sometimes we hit the suggested price range, sometimes we miss. I honestly don't care, and am happy to provide a precedence for those who can't afford to contribute fancy lottery prizes.
With family, grandparents are the only one we need to buy stuff for. My kids (9 and 11) claim they have everything they want. My siblings each have a couple of kids, but we have agreed upon very easy solutions: one of them have more time than cash, the other more cash than time. For the first one, I will transfer a lump sum that they can use for whatever they want (including bills), and they will send us some stuff their kids have made. For the other one, we first talked about sending the same amount of money back and forth, but agreed that would be stupid. So now we'll just skip the transfer entirely, buy something for our own kids, and say it is a gift from auntie.