Wander around San Fran on foot or bike. It's not a huge island, though plenty of hills. There's a lot of gorgeous old houses in certain areas, plus all the predictable areas that people love. I'm not interested in really expensive stores, but Union Square also has ice skating in the winter, maybe that's up?
The pier is all touristy, but often sea lions gather on the back side of it and you can watch them. Plus the people watching the pier allows.
There's some good museums if you're into that, and a decent aquarium. Never went to the zoo, so no idea about that.
Muir Woods was awesome. San Rafael is just north of the city (across Golden Gate Bridge) and many people also like that city. I'm not sure why though, never looked into it.
Then there's Alcatraz Island, if you're interested and willing to pay for the trip.
I'd totally find a good place to sit and watch the ocean for a few hours. If you don't mind cold water, I'm sure you can get down to the water's edge as well and go wading.