In other words, NIMBY
Why should people who are poor, but well behaved, be forced to live near those "troublesome" people any more than you?
Maybe being around annoying people is part of the societal cost, accepting it is part of what the middle class and white people can do to help change the situation, a way to counter-balance some privilege.
That assumes I "owe" society for being middle class and white. Within that context your argument is neat and tidy, true and right. Unfortunately, I don't accept your premise.
This thread seems to have veered off into focusing on the lower economic strata rather than the middle, so I might as well address it.
There have been various suggestions that African-Americans of lower economic classes should value education more. Unfortunately, the level of opportunity that many people assume to be available is not actually present.
Here is a link to a bit of research which shows that 660 of the US' 12000+ high schools (~4.7%) are responsible for half of all African American male dropouts.
http://new.every1graduates.org/wp-content/uploads/2014/06/Methodology-for-Determining-660-Schools.pdfRegardless of where the blame is placed, I contend that students who attend these schools in the country have so little opportunity that even moderate success makes an individual an outlier.
The typical game of political blame tennis shuttles the ball back and forth between schools and parents. I contend that historical (mostly racial, somewhat economic) and current (mostly economic, somewhat racial) segregation is at the root of these endlessly failing institutions.
Government-sponsored segregation left a large number of racially and socially isolated communities that had been systematically deprived of their rights. The various social and environmental ills that other parts of society did not want to deal with were -- and still are -- moved into these communities as a matter of policy. The high-density housing projects, mercury-spewing power plants, homeless shelters, methadone clinics, etc are concentrated in these neighborhoods so that folks who live away from them can enjoy being "middle class" without needing to encounter the world around them or feel that they "owe" anything.
One result is schools with so many problems that no number of social programs can remedy them. These schools have high proportions of students who have suffered acute violence, who are homeless, who suffer housing instability because of parents' employment instability, who were born to parents that used drugs, etc etc. The burden of having all of these difficult students in one place is too much for any combination of teaching and social services to bear.
Blame is often put on teachers. A good teacher can handle a class with 2-3 difficult students. While it is true that the teachers at these schools tend to be less effective than average (teachers who can get jobs somewhere else do!), even SuperTeacher who flies around the room and shoots learning beams from her eyes can't handle a class with 15 students who need special help.
The students who are assigned to these schools are disproportionately African American and Hispanic. I noted earlier that this very small number of schools is responsible for half of African American male dropouts. While students at these schools are no longer formally denied the right to an education, they are effectively being excluded from accessing one of even mediocre quality. Part of their response is to rebel, but their rebellion does not erase the fact that they have effectively been denied an education.
Edit: Expecting kids who go to these schools to succeed is like expecting to grow wealth through stock picking. While a few individuals do exceptionally well this way, they are the outliers rather than the average.