How to keep the cost of pet(s) down? Pretty simple.
First, you're already thinking cat and not dog, so that improves your odds, by and large.
Second, you accept that pets cost money. You have to feed them, at a minimum (duh). You do have an obligation to protect their health by taking them to the vet and getting them immunized. Also, cats are supposed to have their teeth cleaned every so often to prevent diseases that get in via inflamed gums. That's expensive because they have to be anesthetized to clean them. But it extends their lives.
If you get a mixed-breed cat that seems healthy, by and large you're improving your odds that they won't have expensive problems.
Second, don't buy stuff they don't need. This is probably obvious to you. Toys can be cheap. My cats love ponytail holders, the pull-off tabs from gallon jugs and wadded-up tin foil. Sometimes I buy the little mice, but those don't cost very much and they last a while. No silly sweaters or such.
But-- please know that their health is a crap shoot. When and if something happens (fungal infections as a previous poster, bladder stones like my mixed-breed rescue, or any of a myriad of other random problems), then you have difficult decisions to make. How much do you spend on a pet before you decide to put them down? I spent a lot more on my cat last year than I was expecting, because she had bladder stones and needed surgery (about $1300 including all the treatment) -- or I could have had her suffering ended by putting her down. I just couldn't do it. She was only four years old. But I think for someone who is on a tight budget, it's not a monstrous thing to weigh the costs and sometimes make the other decision. Now she's on a special food to prevent more stones from forming. It's a lot more expensive than, say, Iams. And if the stones recur, I'm looking at a really tough decision. How much do you spend to extend a pet's life? Everyone comes to their own conclusion. If you get pets, eventually you will face these things.
So, it's a pretty simple matter to minimize costs for things they don't need, but not that simple to draw the line when they do need something. Personally, I love my cats (I just got a second SPCA cat) and happily spend the money. They improve my quality of life far more than the extra money would.