I'm a 38 year old male, who does not like shopping. I've spent the past 16 or so years trying to figure out how to look presentable at the office and in social situations while not putting in a lot of effort. My conclusion: there is an inverse correlation in this area between effort/time and money spent. So, you have to choose where you're willing to fall along this spectrum. I have completely bought into the MMM philosophy because I want to retire early, but this is one area where I feel that spending a money on a handful of quality items pays off down the road (and perhaps even saves money, nah I can't really back that up, but you don't have to spend an arm and a leg). So, if you're willing to spend a little money (more than at Goodwill certainly), then my advice for spending that money wisely is, (1) find a go-to place where you know the clothes will always fit, don't go out of fashion next week, and match your style, and where the staff are available to help you pick out combinations. (2) Never buy anything unless you know what you will be wearing it with (or you end up with a bunch of crap in your closet that you never wear). And (3) never, ever, buy more than a few items at a time. This last bit of advice is probably the most important; if you buy a bunch of stuff at once you get caught up in the shopping spree and end up at home scratching your head wondering why you bought that one bright orange shirt and why you thought you needed three pairs of pants that match only one other thing in your closet. Don't be afraid to take something back, but having to take something back is sub-optimal.
I've never had much luck with thrift store shopping. My wife has, but I suspect that men keep their clothes until they're rags, whereas women donate their clothes to Goodwill after only one season. So, the contrast at my local Goodwill between men's and women's clothes couldn't be more stark. I'm sure that there are exceptions, so you may as well check out your local thrift store. If you liked shopping, Goodwill wouldn't be a bad idea, because you can go there all the time and find that one diamond in the rough and then tweet about it (#Macklemore #popsometags).
With all that in mind, I use a slightly more expensive, upscale option: Banana Republic. Their staff are helpful, you can mix-and-match their clothes really easily (you can buy a shirt there today and match it with a sweater you bought there six years ago, it's genius). They just make it easy for a guy like me to shop. It's not cheap, but I don't spend that much money on clothes, because I've accumulated everything that I need, and the stuff lasts long enough with gentle use. BR may not be your style (I'm not sure it was mine when I was 25), but just deciding to do all my shopping there has freed up my weekends to do stuff I care about. Maybe some other place will work better for you. The actual store isn't so important as finding one. I guess it wouldn't hurt to check out Target and the like, but other than gym shorts I've not had any luck buying stuff at Target that lasts very long. (My gym shorts from Target have lasted 14 years and counting, I think these things were actually made in the USA, that's how old they are.)
This being MMM, people will suggest that you spend your Saturdays scouring all your local Goodwills for clothes (you would certainly spend less money this way), and bring a female friend along (not sure why a female friend, a fashionable male friend is perhaps the better option since he's actually gone shopping for men's clothes before). But you hate shopping so I suggest that you visit Goodwill once for good measure (maybe your Goodwill kicks my Goodwill's ass), and if that doesn't work out then spend a little money at a store with helpful staff buying something you like and then go spend the rest of your weekend as you normally would (like a bike ride with that female friend that you didn't make go shopping with you).