I have only been to Toronto once, but I loved, loved, loved it! I was there in 2005 when the Air France plane crashed and everyone got out safely. Wow.
Two highlights were taking the ferry & a picnic to Toronto Islands. Since everything had to come by boat, many of the cottages are full of quirks and charm. We stayed at the Intercontinental (fancy!) for the work portion of the trip. The second highlight was the Marriott which is attached to Rogers Stadium, paid with points. Some rooms allow you to watch the game right out your window! We got a free hotel room and a free ballgame, which made our frugal hearts happy. Perhaps you could see one game in person and then another from your hotel room if schedules allow.
I went with a frugal work colleague from the other coast and we called it our Planes, Trains and Automobiles trip. One goal was to use every form of public transportation available. We used a taxi, shuttle, ferry, tram, train, city bus, double decker tour bus, subway, helicopter, airplane and a lot of shank's mare. We had a blast and I'd go again in a heartbeat.
We also did a hop on/hop off bus tour around the city which was fun, because the weather was lovely. We didn't do the City Pass (didn't know about it), but DH and I recently used it in Seattle and thoroughly enjoyed it. Make sure you will be there enough days or it could put too much pressure on you to feel like you got the full value. We had a week. I wouldn't try it for much less than that or that will be all you do.