I buy some organic food at Aldi and Costco. So much cheaper than WF!
Here's what I recommend:
Organic at Aldi: apples, bananas, baby carrots, milk, pasta, stock, applesauce, rice, tinned tomatoes.
Organic at Costco: chicken, minced beef, bread, rice, oats, big carrots, fresh kale, all frozen veg (spinach, kale, broccoli, green beans, edamame, etc), quinoa, all frozen berries, soy milk, eggs, honey, peanut butter, flaxseed, pasta sauce, and much more! I was astonished at how many organic there are when I first visited one here in the States. The UK ones have very few organics.
I don't buy exclusively organic by any means, but these are my top organic bargains, and are truly a fraction of the price at WF. Plus you take your own bags to Aldi so you save on bag wastage, and ethically I think they are comparable, as they stock fewer items and price to sell, so less gets thrown by the stores. They also treat staff pretty well.
I also have a local co-op near me for the occasional other organic purchase. Seriously, ditch Whole Foods and you will halve your grocery bill.
Hope that helps :)