We'd all like to believe that we're enlightened and therefore immune from the many subconscious signals that have a tremendous effect on our attractiveness to the opposite sex. We still have incredibly strong, caveman-era sensitivities to biological signals -- scents/pheromones, body chemistry, visual cues, vocal preferences, on and on. There's no doubt that some women find certain cars sexy, and others not so much (OP's sister as one example).
Now, I 100% agree the OP and anyone else should drive (or bike) whatever they want, and not give a flip what others might think about it, to the extent they can. But we can't fool ourselves that it doesn't matter to how others perceive us, or that "I wouldn't want to attract people who are so shallow they're caring about my car." Some perfectly reasonable, non-shallow women could be influenced by things like the kind of car a guy drives without even being aware of it, just like they're not even aware that the tone of a certain man's voice attracts her, or that broad shoulders do too. If a guy is feeling disadvantaged already with the opposite sex, it might not make sense to "pile on" and drive a car that is clearly not seen as desirable where they live.
I generally don't give a damn what others think about what kind of car I drive or anything else, but it makes sense to at least be aware that driving around a Prius in Lubbock, TX might be a disadvantage with the majority of the opposite sex there, just as driving an F250 truck among hipster women in Greenwich Village, NYC might be similarly disadvantaging.
Not to get into slippery slope arguments too much, but you could replace "kind of car I drive" with style of dress, getting fit or lifting weights, combing your hair, shaving your armpits, whitening your teeth, and so on. Lots of people do those things in part to attract members of the opposite sex and fit in. I'd say the kind of car a guy drives is (and should be) low on the list of things that draws female attention, but again, if you're already disadvantaged in that area, it might make sense to not purposely further disadvantage yourself by picking a car you know is going to turn off, say, 80% of the local preferences.
There's a fine line there somewhere. I could say, "Eh, I'm not interested in any woman that is shallow enough to care about my three foot long beard" and that's okay. But that would be done with a conscious awareness that if I live in LA, I'm likely turning off 99% of prospective mates who would never give a second thought to dating someone that looks like that.
But yeah, the confidence to drive whatever you want is likely going to outweigh 100 to 1 the actual car itself in terms of your attractiveness.