I split my time between two provinces, and the costco advantage varies widely (I'm also living solo and carless, like you).
In Newfoundland, where groceries are expensive and chains are my only option, Costco is a screaming great deal, even for small things (in part because costco prices tend to be fairly consistent across the country, even though it costs more to ship things to NL). I usually go by bus and buy parmesan, tomato paste, olive oil, steel cut oats, bacon, other cheeses, almonds, jeans, socks, occasional meat, and whole-wheat flour. I figure I save about $150 a year.
In Toronto, where there are tons of ethnic grocers and a great monopoly-busting produce distribution network, costco makes much less sense. Supermarket loss leaders and Asian stores are much better deals and involve less travelling.
All that to say, it depends on your local situation!