Money outside retirement accounts probably has gains. You profit on the gains, so the IRS taxes that when you sell.
To illustrate, let's say you bought $10,000 of VFIAX 3 years ago. Vanguard says it's grown 14.5% per year for the past 3 years. So it might have grown +50% in those years to $15,000. If you sold all $15,000 you would have the original $10,000 plus $5,000 of gains. You owe tax on the gains, or $5,000. The average tax payer pays 15% in that case (assets held more than 365 days), or $750. And you don't have to sell it all - you can sell some, instead.
Have you been receiving 1099-DIV from Vanguard every year? When VFIAX issues a dividend, you pay tax on the dividend. It's usually pretty small, like a 2% dividend. On $15,000 that would be $300 dividend, probably also taxed at 15%, or $45 in tax.