To offer a slightly different perspective, I think it's worth using such questions or conversations, which may be rude or tactless, for a bit of self reflection. I was subjected to a similar exchange in the past few years and had a similar reaction at the time--who was this person to critique my generosity, how dare they, etc. But when cooler heads prevailed, I had to admit that they had a point--I actually was overly stingy and had let my drive for savings, frugality and the like blind me to that fact. I've since increased my giving substantially (though it still has further to go!) and, the funny thing is, haven't really noticed any decrease in savings rate.
My point isn't that you should give more--as with most things, it's really an individual and very personal decision--but rather that it's easy to react and take offense at the foolish/ignorant/rude statements of others. In my eyes, trying to critically examine such statements to improve yourself is really the best come back you make to such onslaughts--they use it as an opportunity to display their negative qualities, you use it as an opportunity to further improve your positive ones.
Just my two cents.