I just want to echo what someone said way up about herb gardening. This is counter-intuitive, because herbs don't actually replace grocery costs (unless you were buying a lot of expensive fresh herbs at the store, which we weren't), so let me explain my situation. We moved into a house with a yard after 10 years in apartments, so I decided it was time to start a vegetable garden. I planted about 50 sq ft. of vegetables. It has been an enjoyable learning experience, but the truth is that so far we're not getting a huge bounty. All things considered (counting tools, starts, soil, mulch, etc.), we might be saving $20 a month (results of course may vary).
What I AM surprised by is how much I love my fresh herbs (which currently have only 6 square feet, or 10-15% of the garden - Cilantro, Basil, Oregano, Majoram, Thyme, Sage and Mint).
Fresh herbs make cooking at home so much better. Toss some fresh sage into your black beans and they become something you might buy at a restaurant. Put some thyme or oregano in your omlette (and maybe some collard greens, which I also appreciate), and it becomes a fancy dish. There's just no comparison with the dried stuff.
My point is, consider starting with herbs and seeing how you like it. I kind of wish I hadn't devoted so much space to things like brocolli and radishes (I could buy my whole year's harvest of radishes for less than $10 and the seeds cost $2). Herbs don't save me money directly, but they make eating cheap foods so much more pleasant, which means less dining out, less prepared food, etc.
Theoretically tomatoes would be a good gardening investment, since good organic tomatoes cost a ton, but my tomato plants are not doing well so far, and require more attention than I can give them. Herbs, on the other hand, just seem to grow.