I like this topic as my wife and I discuss this regularly. We are wealthy. Where we live full time, we live in an upper class neighborhood and in an area that is middle/upper middle class in general. So if it's clear you are doing well, nobody really cares. Drive a fancy car? Have plenty of money? Meh. Whatever. So does everyone else. That's our primary residence.
Our secondary residence is in a LCOL rural area. We go to a lodge when up there (similar story as OP, but not a church) and it's VERY low income people there. Lot of poor, working class, lots of sad economic stories and hardship. But also fixed income retired type people. We are quite conscious of concealing our wealth and status when up there. It's not something I would have ever thought of doing being that my primary area has such a different feel to it, and I've always been a "just be who you are" type person. But it's undeniably uncomfortable to stick out as a "rich person" up there, especially to the extent of the our wealth compared to the area.
Basically, as far as being "outed" - it totally depends on where you live and how the local culture is.