When I hear "boyfriend" and "bought a house" my risk antennae go up. Lots can go wrong with owning real estate when you aren't married. Are you on the mortgage? Title? Neither? Do you have a written agreement with him about how the equity is shared and to be disposed of in the event one of you wants out?
Risk can be mathmatically calculated also, and figured into your investment return, so...
This matters because your risk increases. You break up, he wants to keep the house, you don't, he promises to pay the mortgage but the bank won't let you off the note, (and they won't). Now what? You stay on the note and depend on him not to get sick, fired, addicted, married to someone else?
When married people break up, there's lots of legal precedents defining what can happen to split property. For non married couples, not so much.
So my answer to your question would be, pay off the debt as soon as you can to reduce your risk. Bigger picture would be to get married, or straighten out the ownership questions in writing, with an attorney.
Hey, I know, couples never break up ;) and maybe he's your forever guy. I've just heard too many stories of people in a bad spot because of the exact situation you describe. If my daughter asked this I'd be a lot more stern :) IF you want to continue along this, maybe have him quitclaim to you and you refinance only in your name, or vice versa. Sounds like you both make money enough to manage that...Then the other person can invest their money in stock funds for the hopefully shared future while the other builds equity in the home. If you get married, then you pitch it all in the joint ownership pot...