The initial equipment expense is usually greater and like many hobbies it will take some mental effort to stay away from the endlessly increasing cost of 'better' toys, but all grain brewing is the way to go to try to get costs down.
Let's go by ingredient
Water, Malt, Hops, Yeast
Water, some places have access to free cold springs, I did when I lived in Tennessee, collect a bunch of water however you can, let it settle for a few days before brewing, decant off any of the sediment. Avoid buying distilled or packaged spring water. The only thing you truly need to be concerned about is chlorine which will come out with a simple carbon filter. Alternatively, it will simply dissipate if you collect the water and let it sit for at least a day, or you can bring your mash water to a boil first, then let it cool back down to your strike water temp and that should remove enough of the chlorine. If your local water uses chloramine you're just sorta shit out of luck and might need a spring or have to resort to bottled water.
Malt, bulk, bulk, bulk. Look for a local homebrew group, they may do group buys of full bags of malt rather than buying by the pound at a local shop. Ask local brewers if you could possibly buy a couple bags of grain for their cost. Unless you get really good advice or other help from a local homebrew shop, all but a few shops around the country are pretty expensive, provide little selection, and have generally old stock. Mail order can be a great option, buy uncrushed malt for long term storage (easily good for a year). Stick to simple malt bills. Most beers can be made with 3 malts or fewer. Simple American 2-row malt is a great base malt for many English styles, plus it is cheap.
Hops, are expensive. Avoid thinking that you will save money homebrewing yourself Double IPAs. You probably can't, even buying pounds of hops at a time. Most of the time, hops will be sold to homebrewers by the ounce, they will be a few dollars an ounce. There are many beautiful styles of beer that are drinkable, refreshing or comforting (appropriate for changing seasons, and use very little hops. Learn about how hops are used and how they are useful. Your beginning boil charge should be high alpha, it isn't for flavor, just for balancing the sweet malt. Flavor hops go in at 10 or 15 minutes from the end of boil. Aroma hops go in at flameout or after primary fermentation, but like I said, probably avoid styles that need this if you're trying to save money.
Yeast, with some good sanitizer you can easily reuse yeast. For the initial cost You can opt for liquid yeast if you would like, they are expensive, dry yeasts are constantly improving, but most importantly, pick a versatile yeast. If you are going to be brewing a lot of Belgian table beers, wits, etc. Pick a lightly estery Belgian strain. If you are going to be doing the English lineup of browns, milds, pales, etc. Chico is the most widely used but there are other suitable options. Plan your brewing, every two weeks or so, you don't want yeast sitting too long unless you get into more serious yeast storage. Don't use the whole yeast cake, you will be overpitching. make a 500mL starter with some dry malt extract and water, collect some yeast with a sanitized utensil (depends a lot on what you are fermenting in) and put it into the starter a day or two before brewing. Smell the starter before starting the brew, if it smells ok it is clean and you can use it again. After 10 or so uses, beer will be a little more cloudy after fermentation but will still taste fine, you can restart a new yeast strain sometime after this point.
Find a couple recipes you like, preferably something malty with not too many hops and you should be able to do 5 gallon batches of beer for under 15 dollars utility costs included.