Your downpayment doesn’t come into play for taxes, but your numbers don’t add up. If you bought the house for 390K and had a 70K downpayment, then your original mortgage would have been $320K. It doesn’t make sense for there to be $350K remaining.
So if you remortgaged the property to do capital improvements, then you may get to add what you paid for those improvements to the $390K you originally paid for the house to arrive at a higher cost basis, and reduce your gain.
As others mentioned if this was a rental, then you’ll owe depreciation recapture on it. Note that is true wether you actually claimed depreciation on your taxes or not — the IRS just expects that you did your taxes correctly in previous years.