For fun I looked up the last few sales in my neighborhood on zillow, all in late June through July so within the past month or so. Also these are estimates pulled today after these sales have occurred so they should likely be factored in to whatever algorithm it uses.
Prop 1 - Sales price: 575k. Zillow: 526k. ~49k low.
Prop 2 - Sales price: 400k. Zillow: 457k. ~57k high.
Prop 3 - Sales price: 550k. Zillow: 505k. ~ 45k low.
Prop 4 - Sales price: 460k. Zillow: 428k. ~32k low.
So to me they are bordering on useless...they aren't even all consistently high or low...it appears it tries to push everything toward a 'mean' for the neighborhood, thus underestimating the more expensive houses, and overestimating the cheaper ones. Frankly...I could probably have guessed the value tighter than a 100k window (~50k high or low) knowing nothing more than the neighborhood and looking through a few pictures just because I look at all the new listings that come up for my own curiosity.
So that's why I do my own comps....looking through the actual listings one can tell why some places were more than others past a generic size comparison and make adjustments accordingly...some have pools and great landscaping, some have larger lots or ones with great views, one of the houses looked done to the 9's and totally reno'ed, the cheap one looked like crap and was outdated inside, etc.
The best thing about zillow is seeing the data of what is listed and what has sold in a quick format. Their actual estimates are generally crap IMO. The house I just sold was about as standard as you can get and thus should have been perfect for 'zillowing'. Was a decent place but nothing extraordinary about it, almost the exact 'median' house for the neighborhood in terms of size/quality, and in the middle of a HUGE master planned community of over 3000 houses, so there were TONS of great comps...and it was still off by a little over 10% from the final sales price, but it ended up right around where I thought it would and the comps dictated it should when we decided to list it. Had i just gone by the zillow to make that decision we probably wouldn't have even listed.