You know, this "no driving" thing is about you, not about others. What others think about you driving (or not driving) is pretty much irrelevant. If they ask, sure, explain it and tell them how nice it is, but otherwise, unless you want to evangelize bicycles, you don't need to feel good/bad about what others are thinking about your mode of transport.
And, of course, as you've noted, there are perfectly legitimate reasons for using a car. It's a miraculous invention of humankind, let's not deny it. The benefits it has given us are immense. It's just that, like any other tool, there are times when it is useful, and there are times, when it is not. Walking has its place, bicycles have their place and cars have their place (and various modes of public transportation as well, from taxis to airplanes). Use the right tool for the right job, and don't feel ashamed about it.