For Taipei, immediately I think of the National Palace Museum. It's huge, lots of exhibits and very interesting art. Not sure if there many other places in the world with great traditional Chinese art.
Food is really big in Taiwan so definitely use the opportunity to enjoy the huge variety in food. You'll have Din Tai Fung which is good as well as street markets (also good and cheap). Food is pretty cheap and priced well compared to the States and better tasting.
Street markets which I alluded to are also a great place to visit, they can get pretty crowded and buzz with a lot of activity. Snake market I thought was real interesting though your infant might be too young for it or too young to know difference. They also have your standard shopping malls etc but I imagine that's not the reason you are there.
I guess with any city, expect a lot of walking, mass transit, or taxi. Mass transit might be easier for you since there's English available. Cab drivers don't all speak so you might have to show them the chinese characters and say here!
Taipei is pretty safe, safer than most US suburbs and I felt really safe at night there.
It gets real humid and hot during the summer so dress appropriately. Sandals are a must. Umbrellas are also a must.