Look around your area for a store that sells discounted goods. There are chains, like Ocean State Job Lots or Big Lots, and there are smaller/independently owned ones. I make a run to my local discount once every 1-2 weeks to stock up. They sell a lot of junk food, but if you know what you're looking for, they sell a lot of good stuff, too.
The merchandise will vary, but I can almost always get huge supplies of olive oil, coconut oil, canned goods, pasta (often even gluten-free), oats, coffee, tea, soup, dressings and sauces, syrup/honey, and beverages, for 50%-90% off grocery-store price. They also have home accessories, cleaning supplies, and bath and body products. Don't be afraid of expired or damaged goods - that's how you get them cheap, and expiration dates are just a suggestion.
For anyone in the southern New Hampshire/Vermont area, I highly recommend the Mr. G's chain of discount stores (in West Brattleboro, VT and North Walpole, NH).