I never close cards that don't have annual fees. There's no upside, and it would reduce your available credit (and thus raise the % of credit utilized) and also reduce your average length of credit. Both of these affect your credit score.
Personally, though, I'm more of a credit card gamer than you are. I rotate the card I'm using based on signup bonuses, and I end up closing a bunch of cards because they only waive their annual fee for the first year. So my situation is different, but I think the advice still applies.