The big things upon which to splurge -- they'll be used every day (possibly by future children), and having them will make your life easier:
Car seat -- don't buy this used; if it's ever been in a wreck, it could be damaged.
Good crib -- this is perhaps the #1 item to consider for safety since it's an item your baby will use without supervision; you'll also need 2 crib sheets (I loved flannel) and a variety of blankets -- skip the fluffy, never-gonna-be-used comforter. Always make up your crib with TWO (or more) crib sheets and a waterproof sheet between; when the baby throws up or wets the bed in the middle of the night, you just pull off the top sheet and the waterproof sheet . . . and the bed is still "made" -- very convenient in the middle of the night.
High chair -- whether it's a clip-on type, a booster seat, or a full-fledged high chair, it should include a tray that you can remove with one hand; should be able to scoot up to the table without a tray so eventually the child can join the family at the table while still using the chair.
Full-sized stroller that reclines fully -- You may also want a little $20 umbrella stroller, but you'll use the big stroller when the baby is small and can't yet sit up, and you'll continue to use it up until age 3-4 for all-day outings
Other things of a smaller nature:
Clothes -- these seem to last about two days before they're outgrown! Because they're cheap to buy used and because they're so often received as gifts, you'll be able to find an absolute flood of these items at used prices. Also, when your toddler ruins an outfit (yes, it will happen), you'll be glad it cost a dollar.
Toys that existed before the 70s are probably worth buying (and adding onto later): Legos, Lincoln logs, or simple blocks. Baby dolls and accessories. Ride-on toys. Table and chair set. Another good rule of thumb is that you should buy the toy but not all the accessories; for example, buy Barbie, but don't buy her dream house and her Corvette and RV and . . . well, I don't know any other details, but the idea is moderation.
Books -- don't skimp on number or quality, but buy them used. They're available at yard sales for around a quarter apiece. Yes, you can get them from the library, but kids also need to OWN their own books and feel they can return to those "old friends". Babies are ready to begin reading around 4-5 months. Don't read to start with: Just look at the pictures together and talk about colors and pictures. Never, never skimp on your daily reading time.
Photographs -- no one has ever said, "I spent too much time and money taking photographs of my children." You'll love the photographs right away, but you'll love them even more when your kids are older. Keep a good camera handy on a shelf, and grab it often. Think twice about using your cell phone camera; few people ever actually print those pictures. And do have some professional photographs taken. I teach photography, and most people who think they're "as good as a pro" -- aren't. I know two amateurs who are genuinely as good as a pro.
And the breast pump. Whether your insurance pays for it or not, it's not a need. Think about it: How many women in history have nourished their children with breast milk? And how long have breast pumps existed? My kids had formula perhaps 3-4 times each -- ever, and I worked full time from the time they were each about six months old. They nursed first thing in the morning, then had their solid foods during the day, and then nursed again right after school and again at bedtime. They both nursed well past a year. The secrets: Take advantage of help from the nurses in the hospital; breast feeding isn't as instinctive as you might expect -- if you can make it through the first few weeks, you'll be fine. Don't push yourself -- remember that you're learning a new physical skill while you're recovering from an intense medical procedure! You wouldn't be angry with yourself if you couldn't learn to ride a bike the first time you tried -- take it easy on yourself. Drink plenty of water, milk or other healthy drinks -- you can't put out what you aren't taking in.
What you DO need is a breast feeding pillow. Yes, it looks like one of those things that's useless, but it isn't. With my first, I always had a bed pillow lying around on the sofa, and it worked . . . sort of. I'd squish it into the right shape and use it to hold her up. Holding the baby up a bit is a HUGE back-saver. With my second, I bought a "real" breast feeding pillow for $2 at a yard sale -- wow, was it better! It's exactly the right size and shape, and it makes nursing easier. Since you're going to spend HOURS a day nursing, it makes sense to get this little tool to make your life easier.
You'll also need nursing pads to tuck into your bra. I hated the washable ones because they were too thick, and I hated the disposable ones because they contained a thin layer of plastic that held in the moisture and made me uncomfortable. I didn't find the "good ones", I suppose, but they are a need for the first month or two. After that you'll find yourself much more in tune with nursing, and if you feel a sudden "let down", you'll know how to cross your arms and stop the problem before it becomes embarrassing. Always wear print tops when you leave the house.
Food -- Use a small food processor to chop up salt-free veggies and other healthy items. Freeze them in ice cube trays, then pop the food-cubes out into a ziplock bag. You do need to use commercial baby cereals though: Because they're iron-fortified, no real adult alternative exists. It does make sense to buy jarred food for times when you know you'll be out of the house.
Diapers -- yeah, you'll need them. I tried cloth diapers, but I found them to be a monumental amount of work. Part of it was the set-up of my house. Our bathroom was already minimal -- no place for a diaper bucket. Our laundry was in a closet in the kitchen, and we didn't like having the smelly stuff right there between our cooking and eating areas. If I'd been staying at home and if I'd had a real laundry room, I might've been more motivated to try harder. However, I did find a fantastic source for second-quality diapers, which were about half the cost of store brand. Considering the initial investment in cloth diapers and the need to wash in hot water, the cost was actually pretty comparable.