CRAIGSLIST!!!!!
No seriously though, if you live in an area with a good craigslist following you can routinely get FREE landscaping plants and supplies. For plants all you really have to do is go dig them up, that's how I got all of my raspberry bushes.
I'll toss out a non native that for me is super easy to control for low ground cover. Ajuga. gets pretty blue flowers in the spring, grows dense enough to block weeds, likes pretty much all soil types, and has a fairly shallow root system so all I do when it gets out of it's bounds is walk over and scuff up some plants out of the dirt with my shoe, then move them or give them away. But I'll second the users below though, if it's a hill and you want to control erosion, go for fruit plants, blueberries, raspberries etc, at least then you (or more likely the birds) can get something from it.