This is my parents. They re in their 70s, millionaires (perhaps two+ time over, and if not, then certainly close), and still have income that comes close to or maybe exceeds their expenses. Yet I go shipping with my mom and she sees a sweater she loves, but she can't bear to spend $40 on it.
Thankfully, they do allow themselves some splurges. Their house is gorgeous, they just did a minor remodel they've wanted to do since they moved in years ago (cutting a pass-through in a wall, so not a major expense), and they travel, not super cheaply, a couple of times a year. Still, I wish that they would allow themselves a few more "wants", since it seems the wants do exist. Certainly, they aren't unhappy, but... buy the sweater!! But they are frugal by habit.
I worry this will be me, too, especially in the earlier years--that I deprive to the point of feeling, well... deprived because I'm scared.