I'd say the majority of the work done on our house is done by us, no permits. Honestly, they want you to permit EVERYTHING.
So, we permitted the backyard redo, and the upgraded/ new central heat, and the new roof.
We didn't permit the redoing of the plumbing/electrical to add a dishwasher, other plumbing work, window replacement, and insulation.
We *will* permit replacing more plumbing to the street, and if we ever add on (what I wouldn't do for a second toilet) we will permit that too. Whatever we hire out, basically.
My rule is: "can they see it from the outside?" or "is it obvious that something is being done?"