I'm in this situation as well. We could downsize and have a great stash with great income.
But it's a trade off. For some, it's worth it to live in a LCOL area. I'm sure it's doable but I have yet to find an area with the advantages we have in our HCOLA. Unfortunately it costs more to live in a place where all ammenities are walkable, where there's easy access to public transit and transit to airports, where there's lots of parks and easy access to hospitals, libraries, community centres, farmers markets, where there's cultural diversity, people who care about the community, streets that are safe and quiet, left leaning politics, well educated, not homogenously or overly religious, etc.
That's pretty much my list. I'm still looking for a community that has all this at a lower cost, and it could be out there but so far have not found it. It may delay retirement (retiring in 5 years instead of right now) but I don't want to retire to a community that I don't fit in to.
Does this place have to be in the US? I have most of the things you mention in Kuala Lumpur (except possibly for the library bit which requires a US$4-5 ride to get to) - 5 minutes walk to the MRT station (very nice, immaculately clean with pleasant restrooms etc), a straightforward if long public transit ride to the really nice and well connected KLIA (it is more than 40 miles away but easy to get to), hospitals are a short train ride away, pretty nice social clubs are about 30 minutes away by public transit, small farmers markets are there every morning just 5 minutes walking distance away, large ones twice a week within 10 minutes walking distance, a lot of cultural diversity around with Malaysia being itself very racially and culturally diverse and there being large expat communities in the immediate neighborhood (mostly Arab and West African), the streets seem very safe and fairly quiet (there is a strong meme that KL streets are not safe but I have frankly felt more at ease and safe here than, say, Hong Kong where I was last and, of course, Chicago) and the people staying here are fairly well educated as it is a middle class neighborhood but the poorer people who work in the eateries/shops and the farmers coming to the markets are not but have grown up in a very multi-ethnic and cultural environment for centuries.
The politics, however, are a complete mess and pretty much everyone here agrees on that. Some people might not be too happy with the prayers from the mosque which are loud enough to hear with the windows open but the simple expedient of closing the window eliminates that. There are also Hindu and Chinese temples and churches in the neighborhood which is not surprising given the diversity around. There is little overbearing religiosity and beer is freely available at the many 24 hour stores and eateries in the neighborhood and the neighborhood spa has little problem offering "extra services" if you want (and no, I don't use it for that - just the normal massage - but it shows how little effect the official religion has in practice whatever the theory maybe).
For a couple, the cost of living here in my condo which has a nice infinity pool with a very nice sunset view, gym, sauna, badminton courts, a rooftop garden with nice views etc in a fully furnished 3 br/3 ba apartment should be well under US$2k a month and less than US$1.5k if one is willing to eat out most of the time and have the ubiquitous local or West African food. A British friend of mine who is retired spends way less than either figure locally and his expenses only reach the latter figure after including the roughly once a month vacations that he takes. He, however, does live in only a 1 bedroom apartment.