This is my first year biking through the Denver winter. So far the one bit of indispensable gear is my Outdoor Research jacket. I think it's a
Transfer. That jacket rocks - it's my 3-season go-to jacket. I didn't think it would be warm enough for winter biking, but with proper layering and enough biking effort it's just fine into subzero temps. The key is using the drawstring hem and velcro cuffs to lock out the wind - that makes a huge difference.
I also carry a generic pair of waterproof wind pants that pack down to nothing so they can live in my bag. They were really intended for rain and slush protection, but they're pretty warm, too.
Down to about 30 degrees I wear a basic tech-tee, a thin fleece, regular winter hat & gloves, my jacket, and jeans. In colder temps I replace the tech-tee with a stretchy lycra longsleeve top. Below 15 degrees I replace regular gloves with my ski mittens, replace the hat with a fleece balaclava, maybe add long underwear and/or wear the wind pants. That's been good down to subzero in a snowstorm.
Most of this was either stuff I already had (ski gear), was bought cheap (jacket was 50% off at the REI Garage Sale) or from Goodwill (rain pants, lycra top & fleece, $4 each). I'm only going about 3.5 miles each way, so that works for me. Maybe if you're going 10+ miles it might be worth springing for the fancy purpose-made biking gear.