I don't have answers, but some things to think about.
1. If your parents are married, I don't think you can claim one but not the other. Or maybe at all. Check that though.
2. In general, claiming people not your minor children as a dependent is tricky, make sure you get it right per IRS.
3. If you get too much ACA subsidy, I don't think there's a penalty, you just have to pay it back.
4. Does your mom qualify for Medicare? If not, then why did she fail the Medicaid rules? Also, due to ACA, the Medicaid eligibility may have changed in your state, or she could go to the marketplace herself and possibly get a subsidy.
Really, I think you need to talk to a tax professional to make sure everything's set up right and in everyone's best interest. Once you get the tax side settled, then figure out the health insurance side.