I think I beat myself up on this score a little too much sometimes. The wife and I eat out a fair amount, because we enjoy the experience, but sometimes I'm like, "damn, I just spent $60!" (that's for a weeknight couple entrees, couple of beers thing -- a weekend restaurant night is usually closer to, or more than $100). But I console myself with the fact that we're saving close to 50% of our net income (when you include all savings, including 401k + match, savings account, mortgage principle, student loan principle, SS, and retirement contributions). If I include my employer's contribution to my pension -- which is significant -- it shoots up to @70%. Increasing my savings will not affect my FIRE date, which is a function of my retirement/pension eligibility date (and is only 3.5 yrs away), though it would influence my wife's retirement date -- except that she doesn't seem interested in retiring early(ier). She actually likes her job and going to work! :-o She is on track, with no lifestyle changes, to retire at 55 (12 years from now), though we could pretty easily bump that up a few years with some lifestyle changes/more savings on her part, but she just doesn't value the early retire over current lifestyle spending (oh well).
So I do pretty much the same thing -- knowing my goals are met with current savings rate, I consider the rest disposable, and believe me, it's still a ridiculous (non-MMM) amount of money. But the saver/anti-consumer in the back of my mind still thinks I should be cutting expenses/saving even more, though I TRY not to obsess over it, because I don't want it stealing my joy!