Invest. Don't tie up a large portion of your liquid capital into a house. A house is an asset, but it's not liquid - you can't easily get that cash back out. (without selling or renting)
Instead, consider:
1) Max out your 401K contribution to get the dual benefit of saving pre-tax dollars, and reducing taxable income.
2) Max out your HSA account (must have a HDHP to have an HSA account).
3) Max out a T-IRA or ROTH IRA contribution.
4) Invest the $50K in a taxable account.
5) Add $100-to-$200 extra principle payment each month. OR... calculate how much you would need to pay on the mortgage to pay it off just before you reach your FIRE date.
I understand wanting to pay off the house early, but every worker-dollar you sink into a house is a worker-dollar that's not "out there working for you" bringing back growth and dividends.