The bigger question is what's motivating you to still wear Vibrams when all the research has debunked their claims? I actually get angry at any trainer in the gym who wears them because it makes me think these people don't know what they are doing, they are just following a trend that's been disproven. Throw those nasty ugly shoes out and get yourself some of those lightly soled shoes.
Please... walk a mile in someone else's shoes - get it? ;) - before you judge their choices. I have painful bunions and Vibrams have been amazing for me. Before I found them, I was wearing good, solid, sensible shoes (my podiatrist approved of them) that didn't make me look like a weirdo, to no avail. When I switched to Vibrams it was like I was a new person. Went from someone who grumbled about pain after 20 minutes sauntering on flat terrain, to someone who could happily do hour-long walks with my friends' dogs with fun sprints in between. I even went trekking in Nepal. It's not relevant when someone else says they don't do anything, I have experienced how it works for me and that's what matters. Presumably it works for other people who wear them regularly/frequently, too, otherwise they wouldn't do it. The suggestion that people wear them because it's "trendy" is so presumptuous. Everyone thinks they look funny, and they're not always socially acceptable (can't wear it to work, for instance), so if there was a way to get the same benefits in normal-looking shoes, making them versatile for various occasions, for a similar price or less, I'd say most of us would happily go for it. The novelty of their appearance isn't really that big a drawcard, especially in the long run.
But I digress. Back on topic. Which we shouldn't have strayed from in the first place.
To the OP, I've never had this problem with my Vibrams, but I've heard about it. So, I don't know if it helps, but I throw my Vibrams into the washing machine from time to time, and I place them in those fine mesh bags for some protection because I don't think it's good for the shoes to be tumbling around in the machine. In my case, just regular detergent on a hot cycle cleans them to my satisfaction. In your situation, maybe you should wash them more frequently, and also add vinegar and eucalyptus oil to the final rinse?