I've noticed several here have mentioned organic products, and I don't want to turn this into an organics debate, but if you're aiming for the bottom dollar healthy diet I would recommend taking the time to do some real research into organic. (and yes, I already feel your glares)
...
Yeah, organic farming can still use pesticides and herbicides - just not synthetic ones. Just because something is natural it doesn't mean that it's non-toxic. Some natural (non-synthetic) pest/herbicides are apparently worse for the environment than synthetic ones.
For cheap foods:
Meats/Protein
Canned: tuna, Mackerel (makes a great fish pie), sardines. Frozen: Tilapia from Wal-Mart (as someone else mentioned).
Fresh: Eggs. (I don't buy free range because in the US it's little more meaningful than 'organic'. Basically, hens just have to have the potential to reach the outside world - they may be crammed so tightly into a giant barn that they'll never get near the door, though.)
Legumes - dried: red lentils (you can soak in hot water and they will be cooked in minutes or just add to soups etc. while cooking. Very cheap from Indian grocery stores as it's a common staple in their cuisine (dhal)), pinto beans, black beans, chickpeas/garbanzo beans and so on.
Fruit/Vegetables
Bananas (You can freeze any that don't get eaten when perfect, if you're picky, and blend them in smoothies or make
banana 'ice cream' with them), apples
Celery, carrots, onions, cabbage, cucumber
Nuts/Seeds
(Buy in bulk and from ethnic grocers if you can - not in the baking section of the supermarket where they sell 5oz packages as specialty items)
Chia seeds (1 tbsp. per serving so a 2lb $10 jar (
Wal-Mart) is 70 servings with 3g protein and 15g fiber), peanuts/peanut butter, sunflower seeds, flax seeds, almonds (raw).
Grains
Brown rice (more expensive than white but more filling and smaller serve size (1/2c brown vs. 3/4 cup white) purchase at Indian grocery in bulk), store brand unbleached flour (for making bread/baking).
I don't eat red meat or dairy, and tend to avoid wheat mostly, due to digestive issues so I can't be much help there. I think people have already said most of my suggestions but I only skimmed so hopefully there's something original in my suggestions.