A few more things...
Be aware that there are several different ways of accessing tax advantaged accounts (e.g. IRAs, 401(k)s) penalty free before 59.5
I'll link a good thread on that in a minute
Second, Vanguard is one of many brokerages. Others include Fidelity, Schwabb, Scottrade, etc. They are the brokers through whom you invest. Think of them as a grocery store; all will offer basically the same items with some slight differences in pricing and quality (e.g. service). When you open an IRA you will chose a brokerage to buy a particular fund/stock/bond/REIT from. For example, you could choose Vanguard and buy their SP500 index fund (VFINX). Your 401(k) works the same way, only your employer will have chosen the brokerage for you. Note that when you change jobs or retire you can 'roll-over' the funds from your 401(k) into your brokerage (often into your IRA).
Also - you can have BOTH taxable accounts and tax-advantaged accounts with the same brokerage. You can (if you really wanted to), spread your investments over multiple brokerages. The only time that is a good idea is when you have >>$500k and you want to make sure that the FDIC (federal insurance priotecting investments should hte bank go belly up) covers all of your investments.
You aren't there yet, so don't worry about that.
Finally, to address your question, yes the overwhelming reason to invest in IRA and 401(k) (and HSA) accounts is that you will pay substantially less in taxes. Over a few decades it can result in having tens-of-thousands of dollars more (potentially hundreds-of-thousands more) vs investing the exact same amount in taxable accounts.
When you are nearing retirement there are other strategies for reducing your post-retirement taxable burden (i.e. the amount you pay in taxes each year). But that's way down the line. For right now, follow MDM's investment order, make sure you are optimizing your spending ("don't waste money") and you'll be on the fast track towards fianncial independence (FI).
ETA: Here's the link describing how to access tax-advantaged accounts before age 59.5 penalty free. (
link here)
Right now its not super important that you understand all of it - but just be aware that options exist, and so you should shoot to max out your IRA and 401(k) and HSA every year (whenever possible).