After my own 6 month ordeal completely gutting and renovating an in-law apartment behind my house, I think that Papa Bear's breakdown is pretty accurate. A few other tips I've learned to save money through the renovation process:
You can purchase 10% off Lowe's coupons on eBay - this has saved me $100s on cabinets and appliances. If you combine using a coupon with buying through shopping portals (ebates, mrrebates, etc) you can even make a little money. As Papa Bear stated, try to buy bigger appliances around Veteran's Day/Black Friday/After xmas, as box stores are trying to clear out the previous years' inventories.
Check out lighting and furniture through Wayfair's clearance section. I've gotten absurd deals through the site.
For building materials, before hitting the box stores, check to see if you have a Habitat For Humanity store near you. This has also saved me $100s, and you never know what you're going to find each time you visit. I've gotten brand new (still shrink-wrapped) windows left over from a H4H building project for $50-80 (retails close to $250-$300).
In this vein, check and see if you have any salvage stores in your area. We have one here in Maine called Marden's, and their business model is to essentially go to fire and flood damaged stores, clear them out, and resell all the stuff at a huge markdown. I've gotten great deals on flooring, textiles, screws, and random tools.
Regarding windows, there may be a company in your area that makes custom windows from scratch. This sounds expensive, but it actually *can* save you money as you don't have to cover the markup from buying Pella/Andersen windows from another vendor or a box store, PLUS pay for them to install. It's worth at least soliciting a quote.
Regarding paint: depending on the size of your project, almost all box stores with a paint department have a 'rejected paint' area or shelf for returned or inaccurately mixed paints. The cost is usually 80-90% of and may be enough for a project or even a room. You have nothing to lose, especially if the reject color is a perfect match for what you have in mind.
It goes without saying that Craigslist and local Facebook garage sale groups are also great places for deals - both to buy things and to get rid of the things that you are replacing.