I am a CPA licensed in New York. Setting up an LLC in New York is trickier then most other states because New York has a public notice requirement. Filing the forms with the secretary of state costs about $235, but then you have to give public notice of the formation of the LLC, the law was written before the dawn of the internet and has not been updated since, so the criteria for public notice is taking out a classified add in two county wide papers, with one publishing daily and the other weekly for a period of 6 weeks. After 6 weeks the papers issue an affidavit certifying you met the publishing requirement. Then you send the certificates into the secretary of state along with another $50 (gotta love NYS, they get you for everything). At this point you have a properly formed LLC.
As you can imagine the publishing costs can vary widely depending on what county you are publishing in. For an upstate county it can less than $200. For NYC area it can be more than $1500. So in NYS it's the publishing costs that get you.
For tax purposes, as long as the business stays a sole proprietorship - it is considered a single member limited liability company (SMLLC). Single member LLC's are considered "disregarded entities" for tax filing purposes. Thus, no change in the way you file your tax return - still a Schedule C, just with a LLC name on it. There is a $25 "franchise fee/tax" due for the LLC every year (to NYS of course), but that is the only additional tax you need to pay for an LLC.
Does an LLC protect against liability? Or is the shroud of protection as thin as a wet paper bag? That is a question an attorney can answer, but most attorneys I talk to go through the trouble of setting up the LLC for their clients. If the LLC was worthless, then why would they bother? As long as you operate your business appropriately and conduct your transactions "at an arms length" I suspect it becomes much harder to prove a "piercing of the corporate veil". Also, liability insurance and an LLC are not mutually exclusive. Why not both?