I would never limit grandparent time - unless for very serious reasons.
My dad is a super spendypants. The kids get all their big gifts/expensive stuff from him. It does help that my Dad just gives me his credit card and I do most of the gift buying for him (we live very far apart). When he sees them though, he spoils them outrageously. A couple of years ago, we were on vacation with him at Disney. He noticed my DD was very into pin trading. (I had bought a stack of pins for her for $20 from ebay and she had spent years trading them. We never even considered buying new pins). He walked her up to a pin store and told her to buy FIVE pins. She was in shock and my frugal child (13 then) told him she really only wanted ONE special pin. At most, TWO pins. She really didn't want any more than that. He was adamant, saying we were not going to walk away until she chose five pins. She had a super hard time as there were only 2 pins she particularly liked. In the end, she did find 5 pins she liked and she still treasures them. But really, she would have been just as thrilled with one special pin. But that's my Dad.
I do admit I optimize this side of him and try to steer him towards the stuff that the kids need/would like. Backfires sometimes though. We mentioned the kids needed new watches. He showed up with 3 expensive watches each (unfortunately, none in a design they care for so they are stashed somewhere in my house).
Still, thanks to my generous Dad, they had various things throughout their childhood that they loved which would have been out of my normal budget. A swingset one year, a trampoline another year, a train table, etc etc. And I got a couch one year, a basement reno another year, a pergola another time...all kinds of awesome stuff :)