I love to decorate too, but am not willing to pay much--my budget is usually less than $20. I've had great luck with garage sales, Craigslist, and flea markets. Also, I take any and all hand-me-downs from friends and family who redecorate more frequently--they all know I'll take any give-away!
I use cheap vases, like Mason jars, and fill them with interesting odds and ends. One of my favorite decorations right now is an old glass milk bottle my neighbor didn't want anymore. The key for me is not being set on any given item, I have to stay flexible and take what I can find. If I go to a flea market thinking "I need to find a large, rectangular, white-framed mirror," I'm definitely not going to find it. I try to keep an open mind. Consequently, we have a great (in my opinion) smattering of styles throughout our home--mid-century modern, antique, farm-house, hipster...really just a bunch of stuff I like that was cheap/free!
Also, paint is your friend :) I've painted a number of things that were previously quite hideous and now look "retro"--so don't fear badly colored furniture. Have fun with it!