I had the exact same card as you, and when the renewal came up, I called Amex to cancel it, saying that I refused to pay an annual fee. I couldn't get them to give me the card for free (barely used it, so I'm sure I wasn't a high value customer to them), but I "let them" transfer me to another card without a fee (and upped the credit limit a bit while at it), maintaining my credit utilization ratio while not increasing my bills at the same time.
There's no reason to pay a fee on a credit card just to increase your credit limit, ever.