Newborns are pretty inexpensive. We actually saved more money in things we couldn't do anymore than we spend on our son. ;)
The most important thing is to find your way and stop listening to others what they think you should be doing, feeling, saying, whatever. There's not only a whole industry out there that tries to convince you of products and services you must buy, it's also a highly emotional topic even (especially?) with people who do not have kids of their own.
We bought most things used. Kids grow out of not only clothing, but also furniture, toys, etc. so quickly. These things are either completely destroyed or hardly used. There is not in between. Kids do also not need a lot. Our son was as happy to play with toys he got as a present for the family (an unlimited source of stuff flooding your home) as he was with packages, bowls, spoons, empty plastic bottles, etc. He is approaching four now, and he hasn't yet discovered the concept of brands and the "need" of owning what other kids in daycare have. I'm sure that change some day.
The other important skill as a parent is to read and think (harder to do when the newborn is there so make good use of any time before that). There are so many things that seem to make sense initially, but not after some thinking, like most special food for kids (there's baby drinking water) or most of the utilities you can buy. Start by having a need and looking for a solution, instead of buying a solution for a problem you haven't experienced.
There are few things that are really dangerous to a newborn, and those should be pretty obvious: Shaking a baby, feeding vodka instead of milk, dropping it on the floor... Most other things are bothersome, suboptimal, but hardly permanently damaging including feeding (wrong food, breast feeding, not breast feeding), clothes (too warm, too cold), handling (letting cry, waking up, cloth diapers, sleeping on the back, etc.). On most things you can take your time to form your own opinion.