Start sweet potato plants from an organic grocery store potato. One potato/yam will make several plants. There are tutorials online, but I just slice one into 6 or so pieces and put them in small pots of soil in the spring. Last year I tried 2 varieties, and one had 100% success, and 100% of the other variety rotted without sprouting, so it's good to diversify.
I've also planted garlic and regular potatoes from the grocery store successfully. Potatoes are often cited as a dumb thing to grow because they're cheap, but IME, once they're planted, they just keep giving because a couple are usually missed at harvest, and will give you new plants with zero effort the next year. Also, you can grow all the fancy colored and fingerling varieties that are $$ in the store.
Shallots are another good one. I haven't tried starting them from grocery store shallots, but once you plant them, they just keep giving. They'll divide, you harvest and put a % back in the ground and enjoy the rest.
Leeks are also kind of pricey in the store, but they've kind of naturalized in my yard. Let some go to seed and you'll have tons of little leeklets the next spring. Since they're tall and skinny, it's easy to grow other crops amongst the volunteer leeks.
Seeds: yes, save them. Also, most seed packets are good for several years, so no need to buy all new seeds each year.