Craigslist is so awesome here in NYC, and don't forget curbside. Space is at such a premium that, with patience, you can find pretty much whatever you need cheap. We've scored nearly new crockpots, strollers, baby gates, furniture, you name it, for very little money. Public transportation is cheaper than a car; walking and biking are cheaper than public transportation.
+1 on the free entertainment. We do Friday nights picnicking outside of SummerStage, too, and have enjoyed concerts ranging from Jimmy Cliff to The Black Keys. Our UES apartment is under market, and it allows us to enjoy Central Park like our backyard. We use the free city pools, too. Savvy parents sign up for free swimming lessons as well as toy soccer and basketball in Riverside Park. Programming at the library branches is awesome and free, as well (story time, science club, summer reading scavenger hunts, movie nights, etc). Staten Island (!) has good beaches. Really.
In winter, you can skate for free at Bryant Park if you bring your own skates and a lock. Show up early, and it's not crowded. Lasker Rink isn't free, but it's not expensive and no tourists go to Harlem to skate. Get a museum membership and use it to its fullest. AMNH is a good one with kids. We have family days where we're out all day, and we just pack fruit and pb&j's for lunch. We may or may not spring for an inexpensive treat, depending on our budget.
We use our crockpot religiously and entertain friends at our apartment (a junior four) on a weekly basis. Yesterday (snow day) was sledding in CP, followed by chicken soup and grilled cheese for about 10.
Remember that just because *someone* paid top dollar for an item or service, it doesn't mean there isn't a cheaper alternative.