I have a HELOC with a similar balance. The bank it's through has "promotional rates" which are partially amortized withdrawals (meaning the minimum payment is not interest-only and some of the minimum payment goes to principal) at 2.99% for 1 year, 3.79% for 3 years, or 4.25% for 5 years. I can withdraw a specific amount under that "promotional rate" and associated term. My balance is currently entirely under an annual 2.99% rate (used to be 1.99% some years back but gradually increased... 2.25%, 2.49%, etc), and I simply withdraw another set amount out annually to pay down the expiring promotional balance. This is something you may want to look into with your current lender.
Also, I'd check with the 2.99% bank and see if that is indeed a "promotional rate" that resets after a period of time to a rate that is tied to the prime rate (mine is prime + .65). I've never seen a bank offer such a low rate on a long term HELOC (assuming it's a 10-year deal).
FWIW, I've been a proponent of paying down mortgage and HELOC debt for years, but the low rates offered by my bank in the scenario I described has changed my perspective on just how quickly to pay the debt off while using the savings to add to our Vanguard funds.