Great suggestions above. I lived in New Orleans for 4 years and most recently went back to visit for Halloween.
Get a day pass ($3, I think) for the streetcars. Take the green one up St. Charles to see all the non-Mustachian (but pretty and historic) homes.
Walk along the Riverfront area, which goes right past the back of Cafe du Monde, which is a must see (eat) New Orleans sort of thing.
There's also a ferry (free or $1... can't remember) there that goes over to Algiers, which is known for its "gingerbread houses." Another area just to stroll around.
Drinks are 3 for 1 upstairs at some of the bars on Bourbon St., so make the effort to go to the upstairs bar to order.
Lafitte's is definitely worth seeing. Their signature purple drink tastes like Dimetapp, but the bar is a neat place. It's lit entirely by candlelight.
There's a steamboat "cruise" that leaves from the Riverfront. I used to laugh at all the old tourists as my daily run took me by there, but I was dragged onto it by the people I was with a couple of months ago, and it's actually not bad. It's a neat way to see a different side of New Orleans and learn a little bit along the way. I think it was in the $30-45 range, so not exactly cheap, but decent entertainment for a couple of hours.
Wandering around the Quarter, or just sitting somewhere with a drink and people watching, is always interesting. Every time I go there I still find a new and amazing restaurant I've never seen or heard of before.
Central Grocery claims to have the original muffaletta. A "half muff" is plenty for one person, which makes them relatively cheap.
If you're driving, I've heard there's free parking at Harrah's if you stay and gamble for an hour, but I don't know if that's still true.