Best recommendation I can give you is to head to the
forums at toytowngermany.com. Everything you're asking has been covered there over and over. There are also
various wiki articles covering the basics. In particular, in relation to health insurance there are a lot of very useful threads. I haven't been an active member there for a year or two but it really is the place to go for the information you're looking for. With regard to health insurance (and insurance in general, you'll need to check whether or not you'll need Haftpflicht or Hausrat especially - I needed to have proof of both of those to get my rental contract signed) there are two posters in particular JohnG and Starshollow, who put a lot of time and effort into helping with those type of questions. Personally, when I moved here I went for GKV (BKK Wirtschaft & Finanzen) and have been very happy with it. Have heard lots of good things about TKK, especially when it comes to English-speaking assistance. You don't mention whether or not you speak German, so not sure how important that might be for you.
IIRC correctly there is also a thread on what people should bring from US vs. buying new (or second-hand, we are mustachians after all :) ) when you get here.
Banking: I have a current account at Commerzbank and pay no day-to-day fees (contingent on a certain amount being lodged every month). Simply because there was a branch across the road from my apartment and it was easiest at the time, as well as "free". But the online banks do seem to be the best option. ING is another one in addition to DKB and comdirect. I think with ING and DKB at least, getting money from ATMs worldwide isn't an issue. Again, loads of information on toytown about that.
I have prepaid Edeka Mobil (another supermarket) - for 19.95/month I have unlimited calls to other Edeka phones, cheap to other networks and 3GB of data. Have had no major issues and any time I've phoned the customer service they have been really friendly and helpful. I like being able to top-up online, too.
You'll also have to pay a Rundfunkbeitrag (basically this replaced the old television and radio licence a couple of years ago) of about €200 per year - that's compulsory for each household whether you have a TV or not.
Finally, be ruthless in getting your company to help you with anything they can, especially when it comes time to do your taxes. On that note too, if you're not religious make sure they know that so that no church tax is deducted from your salary (and make sure to state it when you register at the town hall). Sorry, maybe you know that kind of thing already, it's sometimes hard to figure out how well people already know stuff about the place they're moving to.