One of my guilty pleasures is watching shows like Kitchen Nightmares, Restaurant Impossible, and Bar Rescue, where experts come in and completely redo the restaurant or bar when it's struggling and on its last legs.
On all three shows, the owners often bought a once successful restaurant or bar that had suddenly turned sour. They try to revive the brand to no avail and then call the experts to come and do the reality show.
Unanimously, the experts often change the name of the bar for basically the same reason you need to change yours--perception can be more important than your actual product, because people need to perceive your business in a good light before they are going to make a purchase.
So ditch the name. Move on and create a new brand. Think about it, don't rush it. Talk to friends and family. Float some ideas. Do a little market research. Then this cloud over your business will be gone.