Ooh! I have experience with a baby in cold! Mine was born last November. The day we brought her home from the hospital it was -35F. I don't know what your temps are, but probably not so extreme?
Everything I read online said that it was perfectly ok to take your baby out for walks as long as it was over freezing. I talked to the doc and was like, "Soooo...April?" He laughed and said, "No, it's perfectly fine to take the baby out, well bundled, throughout winter." My personal cutoff temp was about 0F, though I know there were days I took her out (yes, even as an infant) in colder temps, for short periods of time, like 15-20 minutes. At the time, since she was so little, I had her in: onesie, socks, sleeper with feet, hat, snow suit. I also carried her in the Moby wrap (another layer) to keep her close to my body heat and had a coat which couldn't quite wrap around both of us, but which I could hold over most of her if I thought she was getting cold. She ALWAYS fell asleep on our walks, and she never felt cold when we got back home so she definitely stayed warm enough.
For an older child, especially in a stroller, things should be even easier. What I'm planning for this winter is to bundle her up again--though not as extremely--then put her in the jogger (ours has a cover, so it can keep some of her body heat in) with a heavy blanket or two and perhaps a hot water bottle or rice bag for a little added heat. Doing all of this, I think it's reasonable that I can walk with her all winter here, at least on the days and for the same length of time as I'd normally feel comfortable taking the dog out walking. If it's too snowy/icy for the jogger, I'll put her in the Ergo and wrap a heavy blanket around her, tucking it in so that her feet stay warm. I've done that before and it works well. Her cheeks are the only parts which get chilled, and no more so than mine. Probably less so, actually, since I rub her cheeks fairly often to make sure they stay warm.
There are all kinds of studies showing that being out in nature is good for kids and I don't think that having a little one in a stroller for so long is bad. My girl loves to be out, either in the carrier or the jogger/bike trailer, and I usually use that time to call friends and family because it's one of the few times she's quiet enough for me to do so. :) She takes it all in, and I've had her out for walks of 45 minutes at a time and multiple long-ish walks/rides per day. It's also really nice when she's tired but doesn't really need a nap, just some quiet time. We get home and she's refreshed, happy, and ready to get moving again.
As for the 3 year old, I agree with everyone else that if she's comfortable walking for that distance/length of time, let her. Just make sure she knows that the stroller is always available for her should she get tired.