If anyone ever spent any time on climbing or skiing forums, this has been discussed ad naseum over the years. But since many of the great ski and climbing forums are now defunct (RIP Teletips :-( ) most of that useful information has been lost for ever. I'll put in my few cents worth having lived in a lot of different places....
I've lived in a "mountain town" (South Lake Tahoe, CA), Chicago, Anchorage, AK, Germany, the Bay Area and a few other places. I went to high school in a town of about 7500 people and my dad was/is a well-known figure in town.
There is a big difference between "mountain towns" and plain 'ol small, rural towns. Mountain towns attract people looking for adventure (also some posers who think they are adventerous). Towns that have ski resorts or other seasonal attractions have a much more transient population. Think recent high school or college grads who want to do something fun before getting a "real job" and work at a ski resort for a season or two. Or work at the rafting, mountain bike, etc. company. This is a cool gig when you are young and only need to cover rent, beer and weed money. After a bit, most people realize working for shit wages and a ski pass and eating ramen and easy mac isn't cutting it. Or they start stringing together several seasonal/part time jobs trying to make it work. Some stay, most don't. So you have this revolving door of cheap labor coming and going. We ended up leaving Tahoe for careers. We moved there with jobs we could work remotely and when those contracts ended we did work entry level jobs at the hospital but eventually had to leave for nursing school.
Most cool, mountain towns simply don't have a good economic base to support a thriving, well balanced community. Most jobs are in the leisure/service sector and there might be jobs but the wages suck and housing is un-cheap. Hence you have a lot of population turnover. There are a few exceptions... contractors always seem to do well in towns with lots of absentee homeowners because they rip them off primarily because they can. Tahoe is a great example of this.
Second homes are another part of the equation... Many of the great ski towns of the west are now much less vibrant than they were twenty years ago. Skiing has shifted from a user experience based model (about skiing, duh!) to something else controlled entirely by real estate interests (Read "Downhill Slide" by Clifford for the details). Many great mountain towns have half, or more, oftheir homes sitting vacant 50 weeks out of the year while their owners live in NYC or SF and show up for a week over Christmas and a week or two in July.
This has had a double negative effect on many small ski towns as this drives up the local real estate costs (so "normal" people like plumbers, teachers, mechanics, etc) can no longer afford to live there. And now those homes that once had "normal" people living in them are now empty, so your town has literally had the life sucked out of it.
This has changed a little bit now that some people can live and work remotely, but this doesn't change the price of real estate, making it more affordable for the type of people who make a community feel like a community.
As for plain rural towns, I can't add too much. It's annoying as hell that everyone seems to know everyone's business and there just isn't any anonymity. I can't go anywhere with my dad without him getting into a discussion with someone. That's kinda cute, but also kinda annoying. I only lived there in high school and people don't know me, they ask who I am and why do I look so much like my dad. I go there for deer hunting in the fall, that's about it.
As others have mentioned, towns of 75-200k are about the sweet spot for me. Anchorage, AK was 200k and is the city with the best mountain access in the United States. And it's a city.
I enjoyed living there for five years. In Anchorage, you are always ten minutes away from Alaska.