I used to have ING/Capital One 360, but then switch to Ally. I haven't compared interest rates lately, but the interest rate thing is why I switched to Ally. It was not long after ING changed to being Capital One 360 that I did so, so it's been a while now.
I've only ever had good experiences with both. They pay better interest rates on checking and savings than most banks. With Ally, I can just take a picture of the front and back of any checks I have to deposit and then upload the images online. I also use Ally for automatic bill pay. Whenever I've had to call them for anything there's been little to zero wait to talk to a real person. In comparison, whenever I call Bank of America (one of my credit cards are with them) there's over an hour wait and I just register myself to be called back.
Ally has zero physical locations, so that would be a con if you insist on having a brick and mortar bank to go to. Ally used to offer ATM fee reimbursement if you went to an ATM that wasn't in network, but they don't do that anymore as of last year. The upside is they are teamed up with Allpoint, so there are a lot of ATMs in network.
You can check here to see which ones are near you. I use cash EXTREMELY rarely, so it's not an issue for me.