It sounds like you two need to decide exactly how joint your finances are. As a story for moral support, I had $60,000 in savings of which I put $11,000 in both of our Roths the week after we got married, created a joint emergency fund, and put $30,000 on my husband's student loans. He had $65,000 in debt, which we're now down to about $10,000 a little over a year later, including the $30,000 of "my" savings. For me, joint money really means joint money, and to marry him I was totally willing to "lose" those $30,000 if were to divorce quickly or he dies quickly. Neither of which have happened, nor do I think will happen. So we are firmly living out the "put the savings on the loan" camp right now.
How much would a 20% down payment on a house be? And how much can you two save in a year? I'm a little worried about you putting all your money on your loans while he saves his. You two need a joint budget, even if it means you agree on a minimum amount used to slash loans each month and a minimum amount to add to savings each month. Some amount of that $100,000 is probably more than enough for a down payment unless you live in a seriously high COLA. If that amount is X, could you put all the money above X onto the loans? Or if you're buying a house 2-3 years from now and can save $10,000 a year, could you put all the money in savings above X-20,000 or X-30,000 on the loans? It would save you two the interest costs in a few years.
There are many, many ways to tackle this. If he isn't comfortable with putting "his" money into "your" loans because he wants a joint asset, could you convince him just to pay off one loan this year? Or all the 7-8%? Or everything that puts you above the $2500 of deductible interest expenses per year (if your income doesn't prohibit you from claiming it)? For some people bringing finances together is a process; you guys have to decide what your process looks like.
Also, a caveat. We are looking to purchase a farm that would be high six figures. The bank usually requires a 20% down payment. For the reason of how long it will take us to save that up on top of retirement and emergency fund savings, I regret how much money we put on the loans. Because we're so close we're just going to finish them off this year, but that $30,000 would have been nice liquid cash to have around.