We splurge on cheese as well. I don't like the Norwegian yellow cheese, as it is has no taste. So I buy Dutch yellow cheese, which is not easy to find and not cheap. I try to find the cheapest option, though. Currently we buy it in a shop 20kms from our house. We try to combine the purchase with other errands.
When it comes to blue cheese, there is a great difference between cheese with no taste, bad taste, very little taste or a lot of good taste. I often end up buying a French cheese like Roquefort, which is often available in the normal (cheap) shops. But I'll also do with one of the luxury Norwegian cheeses, which tastes acceptable, just not so strong. This cheese or not often available. All the cheaper cheeses I don't like.
I also buy hard, tasty cheeses like Parmesan or a cheaper copy that tastes acceptable.