I have to assume that this problem has been solved by the French at some point in history.
Seriously though, get a rear rack and some big panniers. (Pannier ironically literally comes from an old French world meaning 'bread bag').
Look for something like this ^ in your pannier. You want it to be deep enough that the bread won't bounce out, but probably with an open top because of the long/awkward shape of baguettes. If it's raining/snowing, you just wrap all the bread in a plastic garbage bag or similar to keep it dry.
Look for something like this in your bike rack:
You want one where there's a lot of tubing to prevent the bag from swinging back into the rear wheel that can be screwed securely onto the bike. Check the weight limit on the rack too . . . you want around 50 lbs at least. (Bread isn't that heavy, but the higher the weight limit typically the more sturdy the rack, and the last thing you want is a rack breaking while out on a ride).
Something to keep in mind while buying a bike is that you want a bicycle with longer chainstays. If the bike has short chainstays your son might strike his heel on the pannier when mounted on his rear rack as he's pedaling.
Pogies are good for keeping your hands warm in the winter if you can get some that fit the bike and don't foul up shifting/braking. They let you wear lighter weight gloves on your hands and would make it easier for your son to hop off the bike, lock it up, deliver the bread, then hop back on and get going again.
If he's doing any cycling at all in the winter I'd also look into getting lights for safety. Even if he's riding in the day, sometimes it can be pretty dark, and snow makes visibility worse. The Cygolite Metro 750 or 600 are both great front lights, and something like a Planet Bike Superflash Turbo or Cygolite Hotshot are decent tail lights.
Also, be aware that if he's riding a bike in the winter and they use salt on the roads, he will chew through components pretty quickly. Expect to replace the chain, cassette, cables, and cable housings and brake pads very quickly (at least once per winter), and be aware that regular cleaning/lubing of the bike will be necessary.
