One of the features of my primary bank (Capital One 360, formerly ING Direct) offers is a read-only Access Code that is specifically intended to be used with sites like Mint. Instead of giving your regular login and password, you give Mint the Access Code instead, which only allows Mint to download transaction data. Even if someone were to hack your Mint account, they'd only have access to view your purchase history. Plus, it's really easy to revoke an Access Code and generate a new one in the event of a breach.
I'm not sure what other banks include this feature, but I've received a few surveys for my other institutions asking me how I felt about their security features and I was sure to mention this as something they should offer.