Housing can be the make or break item for ER. Sure, you may have a paid off place to live, or a cheap place to live that you OWN, but neighbors and neighborhoods can change over time. Assuming quality of life is important to you, how do you plan to deal with this if and when it happens?
Example: My first house I've owned for 5 years now, and the renter behind me changed within 6 months of when I bought the place. Much preferred the old neighbor. Then both of the houses on either side of me foreclosed and have new owners. One of them is a live in flipper, and he now has the place for sale. Basically, the only neighbor still there are the two houses acrossed the street. Heck, even one of the houses next to them the people are about to move on apparently, and they've lived there just a tad longer than I've owned the house I own acrossed the street from them.
Same thing with one of my other rentals, formerly a primary residence, and may be again soon. House acrossed the street was a rental but the owner sold it. Then the house next to me got foreclosed on, and a new neighbor moved in From what I can tell, they are a better neighbor than the neighbor I had before, other than the fact that they are a renter (previous owner was a hoarder with an RV and Semi Truck right at the property line) by my driveway. Then I found out a couple months ago the neighbor two houses down from me who used to have a loud party every summer sold his house and moved after what looks like maybe 5 years or so there, thank goodness.
Point is, sometimes the neighborhood improves, sometimes it doesn't and you never know what you're going to get. I suppose in my case, I can shuffle amongst each of my houses if I get a bad neighbor at one, and wait for them to move on, etc. In my cases above, my neighbors seem to have overall improved. Not to say there isn't still a crummy neighbor nearby, but better than before.
Good thing is, it doesn't appear most people live anywhere for more than a few years, so eventually they will all move on. With a long term outlook, it could be just a matter of waiting them out, or working with code enforcement to keep them in check, etc.