Give yourself a frivolity budget, and stick to it unwaveringly. Allow $30 a month for boots or Starbucks, and allow yourself to save it up if you want to make a larger purchase. If you have a specific dollar amount, then you will likely feel less guilty and it will prevent you from letting "just one pair of boots" turn into Emelda Marco's closet.
Someone asked how long I'd apply this logic. For me, the only debt that is ever okay is mortgage debt (on a residence or rental property). So I'd apply emergency or near emergency budgeting until all other debt was wiped out and I had built up a savings cushion.
However, spending money to save money, as long as the math works out, is a different animal. Buying a soild used bike can save you money in the long run, so that wouldn't be treated at all the same. However, if I was drowning in serious debt that threatened to overwhelm me, I still wouldn't fork out much cash at all for a bike. I'd find whatever I could on freecycle or for very, very cheap elsewhere, and ride that until my head was well above water, and then upgrade. Saving $10/mo on gas does you little good of the $500 you spend on a bike means you can't pay your electric bill.