You talked about fees, but you didn't list your funds or the actual fees you are paying. Start there, find out what total fees you are paying, and any hidden fees your adviser might be receiving. I assume there isn't anything surprising to be found, but now is the time to fully investigate what your current situation is before deciding what to do next.
In general, I think financial advisers error on the side of being too conservative. (People rarely fire advisers for being too conservative, but if you lose money in a given year less knowledgeable investors get mad...) Since you are retired, that might suit you fine, but something to consider given your returns over the last few years. How much risk you want is simply up to you and how much return you are hoping for, but it should be something you are figuring out, not just left to your adviser without your full understanding and input.
So find fee amounts, determine risks acceptance, and then see if you can make your current situation work for you. Sometimes just transferring your funds to lower fee accounts is enough. But transferring to a new company is also fine and isn't that hard. But do a lot of reading here, and read MMM posts on the subject, it is something to really know and feel comfortable with before making your final decision on where to keep you funds.