IIRC, TDAmeritrade's commissions on mutual funds are quite high. Of course, many (most?) Vanguard mutual funds are available as an ETF, and many of them happen to fall on TDAmeritrade's commission-free ETF list.
Whatever you do, keep this in mind: if you have a stock (or ETF) in a brokerage account at Vanguard, and you want to "swap" it for the mutual fund, then you have to sell the stock, wait the full three days for settlement, and only then you can buy the mutual fund. IOW, you incur market risk for the three day settlement. (If you have enough cash, you could wiggle around this.)
What's the harm in keeping two accounts? Keep a Vanguard account for all VG-related funds and ETFs, and keep another brokerage account for non-VG stuff.