You're waaaay overthinking and overly worried about it IMHO.
My Dad is a retired OB/GYN and I grew up in a small town, so he probably treated and/or delivered babies and/or did hysterectomies on most of my friends' mothers, my friends' older sisters, and possibly even some of my female friends.
My sister is a family practice doc, my BIL is a pulmonologist / ER doc / sleep doctor, and my best friend in high school is a urologist. I have a good friend who's husband is an OB/GYN in the same practice my father used to be in, and of course I knew the other doctors who worked with my Dad.
In absolutely all cases, all of the doctors were absolutely professional:
1. They have never (and I believe would never) discuss my or anyone I knew's medical issues or history with anyone. Definitely never anyone outside the family, and only inside the family when I was a minor child and then only with my parents. With HIPAA now, they are even more restrictive with whom they discuss medical matters.
2. This would extend to social gatherings. The fact that this guy is your doctor is nobody else's business, so even mentioning that in a public conversation is not his to bring up because your medical stuff is your private stuff. If you wanted to say "Hey, Doc" at the next soccer game I'm sure he would be polite and say "Hi" back.
3. It doesn't mean they're your friend, though. He may or may not like you personally. He probably has hundreds of patients, and among them are certainly some he doesn't particularly care for on a personal level. That being said, all doctors I know will still provide the best medical care they can regardless of how they feel personally about a patient.
4. I was also trained this way just as a doctor's kid. When I was in high school I volunteered as a candy striper, and we were under strict orders not to discuss anything. If we recognized anyone, again, that was their private business.
5. You also should not, as a previous poster mentioned, be unprofessional to your doctor. It is not appropriate to ask your doctor about a weird mole at your kids' soccer game. Make an appointment with their office and talk about it there.
6. If you're still weirded out about it or if you don't like the doctor for whatever reason, then it's perfectly fine to change doctors. The doctor you change away from won't care and you won't hurt his feelings.
7. By the time doctors become doctors, they've seen a bunch of human bodies doing a bunch of strange, gross, private, weird things, and they've probably answered a bunch of strange / odd / unusual medical questions. After a while it's all just body parts and they really generally aren't impressed / embarrassed / offended / surprised. 99% of the time what you might have going on medically is probably something they've seen a hundred times before. So even though a patient might feel awkward, prostate exams, Pap smears, colonoscopies, breast exams, etc. are all just par for the course for the doctor (even more so for urologists, gastroenterologists, and OB/GYNs).
FWIW to this day I know probably hundreds of women from my home town and I know absolutely zero about which of them (if any) had babies, miscarriages, hysterectomies, STDs, yeast infections, Caesarean sections, or anything else of the sort. Same goes for the patients of my sister, BIL, my best friend, and my friend's husband and all the other doctors in my Dad's former practice.