If I go back to work, I would vastly prefer to be at the grantee-level, though I know it comes with a lot of challenges of its own.
I have worked as all of the above. As a graduate student I was funded by a competitive NSF grant. Then I went to work for a federal science agency in a "soft money" position in which I had to find the funding to support myself and my team. Then I was a hard-funded scientists with a direct congressional allocation.
I thought that the move to being hard-funded would be an improvement, but I was sorely mistaken. Scientists with directly allocated funding still compete for funds, they just compete in a different environment, against other hard-funded scientist who are also after a slice of that finite pie. Soft money scientists have far more freedom to receive funding from a wider variety of sources, and thus have much more control over what they do and how they do it. They're not required to fit into an existing federal funding pipeline, with an overly specific mandate, that already feeds a tank full of other hungry sharks. The amount of politics and bickering and backstabbing and power struggles that I had to endure as a hard-funded scientist absolutely blew me away, and after about three years I gave it up to go back to finding my own funding.
I never had trouble finding money to do the work I wanted to do, because the work I wanted to do was work that I felt was useful and helpful to people, which coincidentally also meant there were lots of people willing to pay for it. I had more money, more autonomy, more people working for me, and more publications as a self-funded competitive scientist than I ever had in my hard money position. I got to write one proposal to fund up to four years of work at a time, instead of enduring an annual budget allocation deathmatch against an ever-changing roster of opponents who would try to discredit my work to make their own look better.
One difference that I did notice in my time as a fed was that the hard-funded scientists were much more likely to work through shutdowns or appropriations lapses, in direct violation of their directives. They were typically people for whom the work was its own reward, and they recognized that the competitive nature of their funding environment didn't allow any time off, because they would fall behind their peers who continued to work. They didn't take vacations, either. Raging assholes, in my experience. Science is supposed to be a cooperative endeavor to advance our collective knowledge, not a cutthroat conflict between practitioners.