No, I mean caulk where the window trim meets the window frame and wall. Usually the trim loosely covers a large gap between the wall's window frame rough-opening and the actual window frame. Sometimes this gap is insulated, sometimes not. Caulking the cracks eliminates myriad small air leaks, which are how heat is moving around. If you seal the floor trim (baseboard) to the floor and wall, as well as ceiling fixture boxes, you can eliminate a lot of convective air movement.
You can use temporary, removable caulk "rope" to seal where window sashes meet the frame, if you have drafts there.
Mostly I wanted to convey that there are many things to do before replacing windows. Sealing drafts, replacing existing weatherstripping on doors and windows, insulation. There are hole in wall bottom and top plates in the underfloor and attic areas that if sealed reduce heat loss as well. Storm windows are also helpful and pretty easy to make. you can buy gaskets to put behind outlet and switch cover plates that also help.