I've made about $10k per year at side hustles and I've never really noticed the taxes. I do not pay estimated taxes. I think we owed some money last year, but not a frightening amount (maybe a few hundred bucks?), and part of the reason was that we changed tax brackets mid-year. The year before, we actually qualified for the Earned Income Tax Credit, so the government gave us back more money than we had paid in! (Mr. FP was a boarding school teacher then, so we had low earnings but free housing. I think any junior faculty with a SAH spouse got the EITC.)
I'm a married lady with a couple of kids, so YMMV. We have all those childcare breaks and exemptions and whatnot.