About the brother: any nearby 'villages' with good accessibility to the units?
This is something that I guess would vary massively depending on location, but I have a friend here who has fairly severe Cerebral Palsy who is able to live alone because of a City Council flat/unit area where it is essentially a miniature retirement village, but not just for the elderly - for anyone who needs an accessible dwelling (ramped doorways, low door handles, wheelchair access bathrooms, low-benched kitchens etc). It is very small, but sufficient for his needs.
He has a caregiver who comes twice a day to help with cleaning and cooking, and anything else he needs help with (putting shoes on, phoning taxi companies to arrange any unexpected rides he needs for the day etc), but he can take care of his personal hygiene needs just fine, he has a full-time job, and he certainly doesn't need round the clock care at all.
I guess what I'm saying is - is there some living arrangement in between a fully-assisted living facility and living in your house with you? I have no idea about your brother's needs, but unless he needs round the clock care, finding him somewhere relatively secure and accessible and paying for a carer to come however often he needs it would be less stressful on your marriage (especially if he's as difficult as you say) - then you can regularly visit and be a brother and friend, and not the bastard who ruins his every waking moment :)
tldr: I wouldn't be bringing him into the family unless there was absolutely no other option. If he's difficult to get along with now, it will likely be worse the sicker and (probably) more resentful he gets. There will be other options, but they are likely highly location specific.