I think you can save money, but you have to be clever about it.
If you like to cook with fresh herbs, growing herbs with save you a mint. If you like tabouli, its dirt cheap to make your own if you grow a reasonable amount of parsley. Basil and pesto is another example you've already alluded to.
For most annual vegetables, to save money, grow from seed, and then save your own seeds. Its not as hard as you might think. I save on tomatoes, pumpkins, corn, beans, snow peas, cucumbers, lettuce, silver beet, chard, beetroot, asian greens, rocket, asparagus when they are in season. We have 4 guinea pigs and i save about $15/week by growing leafy greens of various sorts for them.
Things like potatoes, carrots and onions, you won't save money, unless you want to grow say purple potatoes which cost a fortune in the shops.
The other way to save money is to save on inputs by making your own fertiliser ( bokashi juice, worm wee and castings, comfrey tea, been manure crops) and compost.
PS you can even use diluted urine as long as you don't have a uni/aren't on weird drugs that might affect the soil or its organisms.