Born and raised for a good portion of my childhood in the Bronx. There were, and still are, many nice neighborhoods. Also, property taxes are relatively low, and proximity to NYC is really good, so the Bronx remains an affordable hidden gem for many.
Yonkers, on the other hard, really annoyed me. I lived there when I was first able to work, and I got an extra level of taxation on my income. If you happen to live in NYC and work in Yonkers, you get to pay federal + state + NYC + Yonkers income taxes! INSANE.
Now, probably b/c I lived there and had access to it all the time, I never got what the big deal was. I pretty much hated NYC and went downtown as infrequently as possible. I have heard from my more experienced friends that if you learn where to go and what to do, NYC has a plethora of really good food and experiences for very little or no money. My guess is that any frugal NY-er who lives or spends a fair amount of time there, knows all the tricks and can really stretch their income to enjoy what the city has to offer and still save.