My HSA investment account charges $20/year but once your balance is above a certain value ($10k for mine) the fee goes away. Likewise the savings account charges $3/month until you reach a balance of $5k. Does yours have something similar? If you're not happy with the charges, I think it's possible to roll your existing HSA balance to a less fee-greedy company, even while you're still working for your current employer.
And related to arebelspy's comment about HSA going away, you should be good for 2014 but beyond that is anyone's guess. I'd recommend maxing out your contributions ($6,450/year for a family, $3,250 for an individual).