I live in Yellowknife, where the temperature is usually and steadily around the -35 to -40 mark at this time of the year. And I bike every single day to and from work, and to run errands. Today was a balmy -20, and I was seriously sweating when I got to my office. Here is my easy winter bike set-up. In the real Mustachian way, I did not buy anything I have, including the bike, from a store at retail prices. Most of it is second hand, either new, or in like-new condition. A few things were also free, from Freecycle. I also get most bike parts free from the Good Life Community Bike shop in Calgary. Believe it or not, most big cities in North America have at least one community bike shop with free use space and tools, tons of old parts, greasers ready to help you if you don't know what you are doing, free bike maintenance courses, and lots of great folks to socialize with.
Bike is GT Palomar MTB
Front studded tire. That's all you need, although I am thinking of getting a studded tire for the rear because there is no grip whatsoever as soon as you lift your butt off the seat for a second to climb even the smallest hill when the road is icy or snowy.
Front and rear rechargeable USB lights with flashing function. Not good to see, but ideal to be seen, which is what I want.
Rear plastic fender (useless in winter here)
Handlebar mitts
If you are handy and have the tools, I recommend to repack your freewheel with snowmobile grease to avoid slipping. Also if you live in a relatively flat area, get rid of all useless parts like gears, shifters, derailleurs, etc. and retrofit your bike as a single speed. If you want to go fixie which is my Calgary bike, keep at least one brake. Single speed means less maintenance, and a better workout.
For clothing:
cotton t-shirt
North Face Fleece pullover
North Face soft-shell jacket
North Face Outer waterproof breathable shell
Thin long-john
Axiom Soft shell fleece biking pants. Even at -45, I do not add anything. In wet conditions, I will replace either the fleece pants or the long-johns with Axiom breathable storm biking pants.
One pair of light wool socks
Scarpa hiking boots
Fleece/Neoprene face mask with openings for breathing.
Regular leather gloves with handlebar mitts, or only North Face mitts only when it is warmer -30 instead of using the handlebar mitts.
Snowboarding helmet with ear flaps (I hate to wear it, but I use it because I can attach extra lights at the front and back for visibility.
I never get cold. Never ever, even at the lowest temperature which was -46 sometime during the holidays. Just exhausted when temperature goes below -40, mostly because it becomes very difficult to breathe.
The only thing I am missing is a rack of some sort to put stuff, groceries, etc. Right now i am using a shoulder bag but it is not practical especially when I have to carry heavy things.